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CHAPTER  I 


INTRODUCTION 


Statement  of  the  Problem 

With  the  increased  use  of  classroom  observation  systems,  increased 
concern  is  evident  over  questions  of  (1)  intra-  and  inter-observer  reli- 
ability, (2)  the  instability  of  classroom  behaviors  over  time,  and  (3) 
the  effect  of  the  presence  of  an  observer  in  the  classroom.  Since  reli- 
ability in  the  general  sense  refers  to  consistency  across  a  series  of 
measurements,  the  three  concerns  are  intimately  related  and  almost 
impossible  to  separate. 

Test  reliability  is,  itself,  not  a  unitary  concept.  The  standard 
error  is  used  to  measure  reliability  in  terms  of  the  variability  of  scores 
in  repeated  testing  of  the  same  individuals.  Correlation  is  used  to  ob- 
tain "comparable-forms,"  "split-half ,"  or  "test-retest"  reliability  co- 
efficients in  an  attempt  to  measure  consistency  in  different  kinds  of 
testing  situations  (Cronbach,  1960).  Analysis  of  variance  has  been  shown 
to  give  a  reliability  measure  equivalent  to  that  obtained  with  correla- 
tion adjusted  by  the  Spearman-Brown  formula  (Hoyt,  1955).  Reliability 
always  refers  to  consistency  along  a  series  of  measurements  but  differ- 
ent reliability  coefficients  mean  different  things. 

The  situation  becomes  even  more  complex  when  the  reliability  of 
classroom  observation  systems  is  considered.   Along  with  the  reliability 
of  the  instrument  itself  as  a  measuring  device,  the  reliability  of  the 


recorders  (observers)  must  be  considered.   Intra -observer  reliability, 
i.e.,  the  consistency  with  which  an  observer  records  his  observations, 
as  well  as  inter-observer  reliability,  i.e.,  the  consistency  of  record- 
ing among  different  observers,  must  be  considered. 

As  early  as  1929,  it  was  recognized  that  the  largest  component  in 
the  unreliability  of  observation  systems  is  the  instability  of  behavior 
from  one  observation  to  another  (Barr,  1929).   That  is,  in  order  to 
measure  reliability,  one  must  also  measure  the  actual  changes  of  be- 
havior in  the  classroom,  whether  those  changes  are  random  or  lawful, 
or  due  to  the  presence  of  an  observer. 

Since  the  efficacy  of  any  specific  classroom  observation  system 
must  be  in  doubt  until  questions  of  reliability  are  resolved,  this  study 
was  undertaken  with  the  following  objectives: 

1.  To  estimate  the  intra-  and  inter-observer  reliability  of  the 
Teacher  Practices  Observation  Record,  TPOR  (Brown,  1968),  and 
the  Florida  Climate  and  Control  System,  FLACCS  (Soar,  1971), 

2.  To  determine  whether  any  of  the  behaviors  measured  by  the  TPOR 
and  FLACCS  factors  changed  significantly  over  time  of  day,  and 

3.  To  discover  whether  any  changes  in  behavior  over  the  day  could 
be  related  to  the  observers'  ratings  of  the  effect  of  their 
own  presence  in  the  classroom. 

Discussion  of  the  Instruments 


Teacher  Practices  Observation  Record  -  TPOR 

The  TPOR  was  designed  to  measure  the  extent  to  which  a  teacher  is , 
or  is  not,  behaviorally  in  agreement  with  John  Dewey's  experimentalism. 
Each  of  the  31  pairs  of  odd  and  even  items  represent  contrasting  behaviors 


that  are  traditional  or  experimental  in  nature.   Although  the  TPOR  was 
designed  originally  to  obtain  a  single  score  measuring  agreement  with 
Dewey ,  in  the  Follow  Through  studies  the  items  have  been  factor  analyzed 
to  produce  factors  representing  several  aspects  of  classroom  behavior 
(Soar,  1973).  A  copy  of  the  Teacher  Practices  Observation  Record  is 
included  as  Figure  A-1  in  Appendix  A. 

Florida  Climate  and  Control  System  -  FLACCS 

FLACCS  (Soar,  Soar  and  Ragosta,  1971)  developed  out  of  the  Florida 
Affective  Categories  System  (Soar,  1971)  which  was  a  modification  of  the 
South  Carolina  Observation  Record  (Soar,  1966).   FLACCS  has  two  parts  with 
84-  items  and  78  items  respectively.  The  first  section,  on  classroom 
management ,  is  concerned  with  the  structure  and  control  of  the  classroom 
by  the  teacher  and  aides,  and  with  the  responses  of  the  pupils.  The 
second  section  measures  verbal  and  nonverbal  expressions  of  positive  and 
negative  affect  on  the  part  of  both  teachers  and  pupils.  A  copy  of  FLACCS 
is  included  as  Figure  A- 2  in  Appendix  A. 

Discussion  of.  the  Concepts 
The  reliability  of  a  classroom  observation  instrument  is  dependent 
on  many  things,  not  the  least  of  which  is  the  instrument  itself.  Test 
reliability  is  measured  in  terms  of  the  whole  test,  or  subtests  depend- 
ing on  how  scoring  is  done,  but  not  generally  in  terms  of  individual 
items  which  would  have  less  reliability.   Similarly,  each  of  the  items 
within  an  observation  instrument  may  separately  have  low  reliability, 
but  clusters  or  groups  of  items  may  achieve  the  reliability  that  none 
could  claim  alone.   Data  from  FLACCS,  and  from  the  TPOR  separately,  were 
factor  analyzed  using  principal  axis  factor  extraction  with  multiple  R 


squared  in  the  diagonal,  followed  by  varimax  rotation  (Guertin  and 
Bailey,  1970).  T-scored  items  loading  +_  .40  or  higher  were  pooled  with 
unit  weights  to  produce  incomplete  factor  scores.  Each  factor  purports 
to  measure  some  aspect  of  classroom  behavior,  and  the  factor  score  might 
be  expected  to  have  greater  reliability  than  an  item  score. 

Test  reliability  is  concerned  with  behavior  —  the  behavior  demon- 
strated, by  the  recorded  responses  to  test  questions .  Reliability  in 
classroom  observation  must  be  concerned  not  only  with  the  behavior  of 
the  recorder  (i.e.  observer)  but  with  the  behavior  of  the  teacher  being 
recorded  and  the  interaction  between  the  observer  and  the  teacher.  That 
is,  the  reliability  is  dependent  on  consistency  of  recording  between 
and  within  observers ,  the  stability  of  the  classroom  behaviors  of  teachers 
and  pupils  ,  and  the  effect  of  thj  presence  of  an  observer  on  the  teacher 
and  pupils . 

Intra-  and  Inter-Observer  Reliability 

The  primary  function  in  the  training  of  observers  is  to  educate  them 
to  be  consistent  and  identical  recording  instruments.  To  the  extent  that 
the  observer  is  not  consistent  in  the  recording,  reliability  will  be 
decreased.   If  liking  or  disliking  a  classroom  affects  the  way  the  ob- 
server "sees"  the  behaviors  in  the  classroom,  the  recording  may  reflect 
the  observer's  feelings  rather  than  the  "true"  classroom  behavior.  To 
the  extent  that  events  occurring  in  the  classroom  color  the  way  the  ob- 
server records  ,  reliability  is  decreased.   In  both  of  these  examples  a 
lack  of  consistency  of  recording  within  a  single  observer  decreases  intra- 
observer  reliability.  More  obviously,  to  the  extent  that  two  or  more 
observers  seeing  the  same  behaviors  record  them  differently,  the  inter- 
observer  reliability  of  the  instrument  is  decreased. 


Stability  of  Classroom  Behavior 

Granted  that  one  could  be  sure  of  the  reliability  of  items  within  a 
factor  on  an  instrument  and  sure  of  reliability  within  and  between  ob- 
servers, the  reliability  of  an  observation  instrument  also  depends  on 
the  stability  of  the  behavior  in  the  classroom  if  the  resulting  score 
is  used  as  "typical"  of  the  classroom.   A  measure  is  reliable  to  the 
extent  that  the  average  difference  between  two  measurements  independently 
obtained  in  the  same  classroom  is  smaller  than  the  average  difference 
between  two  measurements  obtained  in  different  classrooms  (Medley  and 
Mitzel,  1953,  p.  250).   Using  this  definition,  all  reliability  data  from 
measures  of  teacher  behavior  over  time  will  reflect  real  differences  in 
teacher  behavior  from  one  observation  to  another. 

Classroom  behaviors  in  kindergarten,  first  and  second  grades  in- 
clude activities  ranging  from  free  play  to  very  structured  situations, 
from  active  game-playing  to  listening  to  stories,  from  show-and-tell  to 
learning  to  read.  Such  variability  of  classroom  situations  could 
contribute  to  a  lack  of  stability  in  the  measurement  of  classroom 
behaviors .   '         . 

Presence  of  Observers 


It  has  already  been  stated  that  events  in  the  classroom  which  affect 
the  way  the  observer  records  diminish  reliability.  Additionally,  to  the 
extent  that  the  presence  of  the  observer  differentially  affects  the 
behavior  in  various  classrooms,  reliability  decreases.   If  one  teacher 
is  relatively  unaffected  by  the  presence  of  observers,  while  another  who 
is  apprehensive  at  first  gradually  relaxes,  and  a  third  becomes  more  and 
more  upset,  their  differing  responses  to  being  observed  may  be  reflected 
in  the  behaviors  under  observation,  causing  decreased  reliability. 


Significance  for  Educational  Research 
Knowledge  about  reliability  relative  to  several  groups  of  behaviors 
(i.e.,  factor's)  on  two  instruments  measuring  different  aspects  of  class- 
room behavior  has  relevance  beyond  knowledge  about  the  two  instruments 
themselves.  Quality  control  is  as  necessary  in  research  as  in  industry, 
and  awareness  of  influences  tending  to  decrease  reliability  can  lead  to 
methods  of  overcoming  the  deficiency . 

Knowledge  of  intra-  and  inter-observer  reliability  may  change  methods 
of  training.  Knowing  the  stability  of  different  kinds  of  behaviors  and 
the  effect  an  observer  might  have  on  the  classroom  has  implications  for 
deciding  the  length  of  time  to  be  spent  in  a  classroom  visit.  What  we 
learn  from  a  closer  look  at  the  various  aspects  of  reliability  may  hope- 
fully help  to  make  the  observation  systems  better,  the  training  of  ob- 
servers better,  or  the  data  processing  more  exact,  while  increasing  our 
awareness  of  classroom  observation  as  an  interactive  process . 


CHAPTER  II 


REVIEW  OF  THE  LITERATURE 


Classroom  Observation 


Classroom  observation  is  not  a  new  procedure.  What  is  relatively- 
new  is  an  attempt  to  measure  the  teaching  process  through  systematic 
observation  techniques  and  to  relate  those  process  measures  to  the 
growth  of  students.   Morsh  and  Wilder  (1954)  found  that  attempts  to 
validate  process  criteria  by  correlating  them  with  measured  pupil  growth 
had  been,  up  to  that  time,  generally  unsuccessful.  The  development  of 
Flanders'  Interaction  Analysis  opened  a  new  avenue  of  research  (Flanders, 
1965,  1967,  1970).   In  the  proliferation  of  observation  systems  since 
that  time,  the  single  instrument  that  has  been  most  often  used  as  the 
basis  for  an  expanded  system  or  as  part  of  the  synthesis  of  old  systems 
into  new  ones  is  Flanders'  (Rosenshine  and  Furst,  1973,  p.  143).  The 
Observation  Schedule  and  Record  (Medley  and  Mitzel,  1958,  b)  also  led 
to  large  amounts  of  process-product  research. 

Government  support  for  the  evaluation  of  large  scale  education  pro- 
grams ,  such  as  Head  Start  and  Follow  Through,  has  also  encouraged  the  use 
of  systematic  observational  techniques .   Assessment  of  program  differ- 
ences in  the  Planned  Variation  phase  of  Head  Start  and  Follow  Through  has 
been  accompanied  by  attempts,  not  only  to  collect  observation  data,  but 
in  some  cases,  to  relate  observation  data  to  the  growth  of  pupils. 
Systematic  observation  instruments  which  have  been  used  in  Head  Start 


and  Follow  Through  evaluation  include  the  Personal  Record  of  School  Ex- 
perience -  PROSE  (Medley,  1968),  The  Teacher  Behavior  Survey  Instrument 
and  the  Child  Behavior  Survey  Instrument  (Katz,  1968;  reported  also  by 
Gordon,  1973),  Stanford  Research  Institute's  Classroom  Observation 
Instrument  -  COI  (Stallings,  1972),  the  Teacher  Practices  Observation 
Record  -  TPOR  (Brown,  1968;  Soar,  1971,  1973),  the  Reciprocal  Categories 
System  -.RCS  (Ober,  et  al..  1968;  Soar,  1971,  1973),  the  K-1  version  of 
the  Taxonomy  of  Cognitive  Behavior  (Ragosta,  et  al.,  1971;  Soar,  1971, 
1973),  and  the  Florida  Climate  and  Control  System  -  FLACCS  (Soar,  1971, 
1973). 

Much  of  the  data  from  Head  Start  and  Follow  Through  are  not  yet 
generally  available.  One  can  only  hope  that  the  research  involving  ob- 
servation svstems  might  be  further  explicated  and  the  best  of  the  systems 
used  to  replicate  and  add  to  the  findings . 

The  orderly  growth  of  knowledge  might  be  promoted  if 
researchers  were  less  ready  to  devise  their  own 
observation  systems  and  instead  worked  more  closely 
within  existing  systems,   (Neujahr,  1972,  p.  228) 

Validity  and  Reliability 

Validity 

Dopyera  and  Lav  (1969),  in  surveying  the  procedures  for  assessing 
the  program  environments  of  Head  Start  and  other  pre-school  children 
raise,  among  other  issues,  the  metric  problems  of  reliability  and  validi- 
ty. The  validity  advocated  in  the  addendum  to  their  final  report  is 
predictive  validity  (i.e.,  association  of  a  behavioral  measure  with 
independent  child -change  measures).  Although  some  would  oppose  this 
position  as  being  premature  (Jackson,  1967),  support  for  this  definition 
of  validity  is  high.  Johnson  and  Bolstad  (1973),  in  writing  of  the 


validity  of  naturalistic  behavioral  data,  stress  the  need  for  establish- 
ing predictive  validity.  Gage  (1967)  considers  an  educational  study  a 
bit  of  research  only  when  category  development  and  use  is  accompanied 
by  achievement  measures.  Rosenshine  (1971)  reports  on  approximately 
30  studies  in  which  data  from  systematic  observation  had  been  related 
to  student  achievement  or  growth.   Indeed,  Rosenshine  and  Furst  (1973) 
advocate  an  emphasis  on  predictive  validity  before  reliability. 

The  predictive  validity  may  diminish  or  increase  as  soon 
as  reliable  observational  data  are  used  in  a  statistical 
analysis ,  but  at  least  the  researcher  would  know  whether 
the  variables  are  worth  studying.   (Rosenshine  and  Furst, 
1973,  p.  169) 

Many  developers  of  observation  systems  overemphasize  observer  agree- 
ment at  the  expense  of  validity.  Medley  and  Mitzel  recognized  this 
as  early  as  1963  when  they  noted  that  concern  with  observer  agreement 
may  have  eliminated  valid  items  from  an  observation  instrument  (Medley 
and  Mitzel,  1963,  p.  263  f f ) .  Additional  evidence  in  this  regard  comes 
from  researchers  in  behavior  modification. 

Finally,  high  reliability  bears  no  relation  to  the  validity 
of  a  particular  measure.   It  seems  that  reliability  can  be 
purchased  at  the  expense  of  the  substantive  meaning  of  a 
particular  measurement.   (O'Leary  and  Kent,  1973,  p.  88) 

Reliability 

Different  kinds  of  reliability  measures  mean  different  things.   In 
test  reliability  both  the  standard  error  and  correlation  have  been  ad- 
vocated as  two  differing  approaches  to  the  problem  of  reliability 
(Thorndike,  1950).   Correlation  has  been  used  to  obtain  coefficients  of 
stability,  or  coefficients  of  equivalence  and  stability  depending  on 
the  kind  of  test  data  being  correlated  (Cronbach,  1960).   Analysis  of 
variance  has  been  shown  to  give  the  same  reliability  measure  as 
correlation  adjusted  by  the  Spearman-Brown  formula  (Hoyt,  1955). 
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With  classroom  observational  measures  reliability  becomes  more  com- 
plex.  It  has  often  been  reported  in  terms  of  agreement  between  observers 
(Weick,  1968;  Flanders,  1965,  1967).   In  the  Head  Start  and  Follow  Through 
studies  mentioned  earlier,  percent  agreement  was  used  by  both  Katz  and 
Stallings.   In  one  case  the  formula  used  was: 

#   of  agreements 

#  of  agreements  +  #  of  disagreements 

where  non-occurrence  of  a  behavior  was  included  in  the  calculations  (Katz, 

1968).  Two  trained  observers  in  that  study  were  able  to  achieve  an  8U.6 

percent  overall  percentage  of  agreement.   Stallings  (1972)  reports  a 

mean  percent  agreement  of  86.8  percent  for  variables  least  frequently 

recorded  and  69.0  percent  for  variables  most  frequently  reported,  based 

on  eight  pairings  of  observers  in  different  locations. 

The  question  of  when  percent  of  agreement  f itures  are  "good"  or 
"good  enough"  is  a  difficult  question  to  answer.  There  are  no  establish- 
ed criteria  and  very  little  data  by  which  a  relative  evaluation  could 
be  made.  Johnson  arid  Bolstad  (1973)  speak  of  80-85  percent  agreement 
as  traditionally  computed  as  a  realistic  upper  limit  for  their  complex 
coding  in  naturalistic  observation. 

Reliability  figures  defined  in  terms  of  percent  agreement  have 

generally  been  based  on  the  total  data  collected  rather  than  the  data 

as  used  in  further  analyses. 

The  reliability  figures  reported  [for  Flanders'  Interaction 
Analysis]  are  based  on  total  data  matrices  and  do  not 
represent  interrater  reliability  for  subscores  and  subsections 
of  the  matrix  that  are  employed  in  recommended  data  analyses . 
(Mitchell,  1969,  p.  710) 
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Johnson  and  Bolstad  (1973)  acknowledge  percent  agreement  as  a  basis 

for  one  kind  of  reliability  but  propose  one  central  prescription  -  the 

agreement  data  should  be  computed  on  the  score  used  as  the  dependent 

variable.  They  see  no  sense  in  reporting  overall  average  agreement  data 

except  perhaps  as  a  bow  to  tradition.  Confirmation  for  this  point  of 

view  comes  from  data  out  of  the  National  Evaluation  of  Follow  Through . 

The  data  do  illustrate  the  inadequacy  of  presenting 
reliability  in  terms  of  observer  agreement  for  a 
total  instrument  which  is  common.  The  RCS  data,  for 
example,  indicate  that  reliabilities  of  separate 
measures  ranged  from  indicating  doubtful  value  to 
ones  that  were  quite  high.  To  cite  one  overall  value 
for  the  instrument  as  a  whole  does  not  seem  to  be 
useful.   (Soar,  1973,  p.  129) 

In  addition  to  percent  agreement,  correlation  has  been  used  in  Head 
Start  and  Follow  Through  observa"^ional  data  to  report  reliabilities.   In 
1968  a  pilot  study  was  conducted  using  a  preliminary  form  of  PROSE 
(Medley,  1968).  The  reliabilities  for  all  67  alternatives  were  re- 
ported by  correlating  the  total  number  of  times  an  item  was  checked  by 
one  observer  with  the  total  number  of  times  it  was  checked  by  the  other 
observer  in  four  visits  to  classrooms  involving  27  pupils.  The  correla- 
tion was  stepped  up  by  the  Spearman-Brown  formula  to  obtain  the  relia- 
bility of  the  total  observation  time.   Many  of  the  reliabilities  were 
significant;  some  were  close  to  zero.   Taken  as  a  whole.  Medley  felt 
the  data  clearly  indicated  that  PROSE  records  made  by  different  re- 
corders at  different  times  detected  stable  differences  between  pupils 
and  teachers.   However,  Medley  recommended  that  reliability  of  scales  -- 
not  items  —  be  used  in  the  Head  Start  Longitudinal  study  itself. 

Reliabilities  for  FLACCS  and  TPOR  were  calculated  from  limited  data 
(Soar,  1973).   The  16  observers  who  collected  the  actual  classroom  data 
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observed  a  video  tape  of  one  teacher  before  beginning  their  winter  ob- 
servations and  a  tape  of  another  teacher  at  the  end  of  their  field  work. 
Data  from  these  observations  were  used  to  estimate  reliability  by  Hoyt's 
(1955)  analysis  of  variance  procedure.   In  his  discussion.  Soar  points 
out  that  reliabilities  of  some  factors  may  have  been  overestimated  while 
other  estimates  were  meaningless  because  of  the  limitations  of  the  use 
of  video  tape. 

All  of  the  measures  of  reliability  mentioned  above  are  concerned 
with  inter-observer  (between-observer)  reliability,  i.e.,  whether  all 
observers  are  equivalent  recording  instruments.   Brown,  et  al.,  (1968)  was 
more  concerned  with  intra-observer  (within  observer)  reliability,  i.e., 
whether  an  observer  is  consistent  over  time  in  recording  his  observa- 
tions.  In  a  lucid  article  discussing  reliability,  he  reports  both  inter- 
and  intra-observer  reliabilities  for  untrained  observers  using  the  TPOR. 

The  inadequacy  of  most  of  the  reported  measures  as  indices  of  re- 
liability has  been  discussed  widely  (Medley  and  Mitzel,  1963;  Westbury, 
1967;  Dopyera  and  Lay,  1969;  McGaw,  et  al.,  1972;  Soar,  1973).   Indeed, 
Medley  and  Mitzel  have  distinguished  three  coefficients:  the  reliability 
coefficient,  based  on  observations  made  by  different  observers  at  differ- 
ent times;  the  coefficient  of  observer  agreement,  based  on  observations 
made  by  different  observers  at  the  same  time;  and  the  stability  co- 
efficient based  on  observations  by  the  same  observer  at  different  times. 
They  place  less  importance  on  observer  agreement  and  more  importance  on 
the  reliability  coefficient  which  best  estimates  the  accuracy  of  measure- 
ments .•  They  find  it  wasteful  to  send  two  observers  into  the  classroom 
to  measure  the  same  behavior,  preferring  to  have  two  estimates  of  be- 
havior by  using  the  observers  at  two  different  times.   Westbury  (1967), 
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reporting  a  lack  of  stability  in  teacher  interaction  patterns,  feels 

that  more  is  needed  for  instrument  reliability  than  a  measure  of  observer 

agreement.  Others  agree: 

Establishing  that  two  or  more  observers  agree  a  large 
portion  of  the  time  is  a  reasonable  approach  when  that 
figure  is  an  unbiased  estimate  of  the  overt  occurrence  of 
phenomena.   Unfortunately,  the  fact  is  that  many,  if  not 
most,  reliability  estimates  presented  in  this  fashion  are 
,  not,  in  fact,  unbiased. .. (Dopy era  and  Lay*,  1969,  p.  51) 

O'Leary  and  Kent  (1973)  discuss  the  reliability  of  observers  using 
O'Leary's  nine  category  observational  code  for  measuring  the  disruptive 
behavior  of  children  in  a  classroom  setting.  They  report  higher  re- 
liabilities when  the  trainer  was  absent  than  when  he  was  present  during 
observations  and  higher  reliabilities  as  computed  by  the  observers  than 
reliabilities  computed  by  the  experimenter.  Several  studies  report  ob- 
server bias  -  a  drift  in  recording  to  match  those  with  whom  one  is  paired 
(O'Leary  and  Kent,  1973;  Skindrud,  1973).  Such  data  suggest  that  indi- 
vidual observers  or  groups  of  observers  should  not  be  confounded  with 
different  experimental  treatments  but  should  be  distributed  across 
treatments . 

In  view  of  the  many  problems  associated  with  the  concept  of  re- 
liability, particular  attention  must  be  paid  to  the  usefulness  of  the 
types  of  reliability  reported  in  observational  data.  The  timing  of  re- 
liability data  is  also  an  important  factor.  Many  studies  report  reli- 
abilities on  data  collected  before  the  instrument  is  actually  used  in  a 
study.   Such  reliability  data  is  usually  obtained  during  the  training 
of  observers.   Reliabilities  on  FLACCS  and  TPOR  as  well  as  PROSE  fall 
into  this  category  although  in  the  case  of  FLACCS  and  TPOR  additional 
reliability  data  was  collected  after  the  major  data  collection  effort 
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(Soar,  1973;  Medley,  1968).  Reliability  data  is  sometimes  obtained  dur- 
ing the  actual  data  collection.  The  COI  reliability  data  were  collected 
by  having  a  pair  of  observers  make  simultaneous  recordings  in  eight 
classrooms  during  the  actual  data  collection  effort  (Stallings,  1972). 
Medley  and  Mitzel  (1963)  approve  the  measurement  of  reliability  by 
an  analysis  of  variance  in  the  actual  data  analysis  phase.  Dopyera  and 
Lay  (1969)  reemphasize  that  approach  as  do  McGaw,  Wardrop,  and  Bunda 
(1972)  although,  to  date,  no  such  reliabilities  have  been  reported  for 
observational  data  in  Head  Start  or  Follow  Through. 

Stability  of  Classroom  Behavior 

As  early  as  1929,  it  was  believed  that  the  instability  of  teacher 
behavior  from  one  lesson  to  another  was  the  major  component  in  the  unre- 
liability of  observation  systems  (Barr,  1929).  But  findings  have  been 
mixed,  with  some  behaviors  seemingly  stable,  others  not  stable. 

Jersild  (1939)  used  an  observation  scheme  tallying  pupil  behaviors 
in  seven  categories:   cooperative,  critical,  leadership,  work-spirit, 
experimental,  recitational,  and  self -initiating.  Pairs  of  observers 
visited  teachers  ten  times  for  half  an  hoiir  each  visit.   Stability  of  be- 
havior was  measured  by  correlating  the  sum  of  the  first  five  half-hour 
observations  of  each  teacher  with  the  sum  of  the  last  five  observations 
for  each  category  of  behavior,  adjusted  by  the  Spearman-Brown  formula. 
The  first  four  categories  mentioned  above  had  stability  coefficients  of 
.88,  .82,  .90,  and  .94,  respectively,  although  the  percent  agreement  be- 
tween pairs  of  observers  for  the  same  data  were  70,  70,  76,  and  43. 
Lower  stability  coefficients  (.44,  .68,  and  .50,  respectively)  were  re- 
ported for  the  last  three  categories  although  their  percents  of  agreement 
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were  higher  (87,  83,  and  77,  respectively).  Thus,  student  behaviors 
cavegorized  by  Jersild  as  cooperative,  critical,  leadership  and  work- 
spirit  appear  to  be  relatively  stable  over  time  while  those  categorized 
as  experimental,  recitationa,  or  self-initiated  are  less  stable. 

Hughes  (1959),  on  the  basis  of  a  small  study  involving  six  teachers 
who  were  observed  during  four  half -hour  sessions,  concluded  that  teacher 
behavior  patterns  are  stable  over  time.  Her  data  were  based  upon  an 
average  of  250  agreed-upon  teacher  acts  during  each  half -hour  observation 
period.  All  agreed-upon  acts  were  tallied  as  frequencies  in  seven  cate- 
gories:  controlling  functions,  imposition,  facilitating,  content  de- 
velopment, personal  response,  positive  affectivity,  and  negative  affecti- 
vity.  Hughes  felt  that  the  number  of  controlling  acts  exhibited  by  a 
given  teacher  in  different  situations  does  not  vary  significantly  when 
compared  with  the  interaction  between  situations  and  teachers. 

Soar  (1973)  reports  on  the  stability  of  behavior  in  17  Follow  Through 
classrooms  observed  for  one  day  each  (12  five-minute  time  samples  during 
the  day)  during  the  first  week  of  school  in  September,  again  in  late 
October  or  early  November,  and  finally  again  during  January,  February,  or 
March.  There  were  no  significant  differences  across  time  for  six  of  the 
nine  factors  of  the  Florida  Climate  and  Control  System,  FLACCS,  or  for 
any  of  the  seven  factors  of  the  Teacher  Practices  Observation  Record, 
TPOR.   The  exceptions  were  interesting.   FLACCS  1,  Strong  Control,  had 
means  of  52.1,  49.8,  and  48.5  for  the  three  time  periods  while  Factor  7, 
Pupil  Negative  Affect,  had  means  of  54.8,  53.2,  and  49.7.   Both  factors 
had  significant  differences  over  time.  One  other  factor  demonstrated  a 
significant  difference.   Factor  6,  Work  Without  Teacher  had  means  of 
49.3,  53.1,  and  50.7  and  was  statistically  significant. 
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Bellack,  et  al.,  (1965)  reports  an  astonishing  degree  of  stability 
over  time  in  the  pattern  of  discourse  both  for  teachers  and  for  pupils. 

Mitzel  and  Rabinowitz  (1953)  and  Medley  and  Mitzel  (1958a)  on  the 
other  hand,  found  that  the  variation  in  teacher  verbal  behavior  from 
observation  to  observation  provided  a  major  source  of  variability.  Ob- 
servers in  that  study  were  self  trained  on  Withall's  technique  (Withall, 
1949)  categorizing  all  teacher  statements  as  (1)  learner-supportive, 
(2)  accepting  or  clarifying,  (3)  problem  structuring,  (4)  neutral,  (5) 
directive,  (6)  reproving  or  disapproving,  or  (7)  teacher-supportive. 
Categorization  in  this  technique  is  based  not  on  the  content  of  the  state- 
ment but  on  the  intent  of  the  teacher  making  it.   The  index  used  in  this 
study  was  the  sum  of  statements  in  categories  1-3  (learner-centered 
statements)  divided  by  the  total  number  of  statements  for  each  of  four 
teachers  by  each  of  two  observers  on  each  of  eight  visits. 

The  significant  effect  for  teacher-by-visit  is  an  answer 
to  the  questions  posed  in  this  study  concerning  differences 
among  teachers,  and  stability  of  teachers'  verbal  behavior 
from  time  to  time.   It  was  concluded  that  teachers  showed 
marked  fluctuations  in  their  verbal  behavior;  "real"  dif- 
ferences among  teachers  were  apparent  in  spite  of  these 
fluctuations.   (Mitzel  and  Rabinowitz,  1953,  p.  15) 

McGaw,  et  al.,  (1972)  state  that  variability  of  the  object  of  ob- 
servation is  the  most  important  source  of  error  variance,  and  unless 
stable  estimates  can  be  obtained  variability  across  classrooms  or 
teachers  will  inevitably  be  swamped  by  variability  within.   They  take 
issue  with  the  Medley  and  Mitzel  (1963)  position  that  instability  of 
teacher  behavior  is  due  to  random  error  in  the  person,  environment  or 
both . 
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This  implies  that  the  characteristic  being  measured  is 
stable  in  a  sense  that  does  not  allow  of  lawful  change. 
While  this  may  be  a  reasonable  assumption  in  relation  to 
relatively  enduring  aspects  of  personality,  it  is  un- 
reasonable when  other  types  of  behavior  patterns  are  being 
observed.   (McGaw,  Wardrop,  and  Bunda,  1972,  p.  15) 


A  lack  of  knowledge  and  need  for  more  research  on  the  stability  of  te; 
er  behavior  has  been  pointed  out  repeatedly  (Rosenshine,  1970;  McGaw, 
et  al.,  1972). 

Observer  Effects 


Medley  and  Mitzel  (1953)  point  out  that  one  of  the  reasons  given 
for  failing  to  include  classroom  observation  in  research  studies  is  that 
the  presence  of  an  observer  in  the  classroom  is  so  disturbing  that  the 
behavior  observed  cannot  be  regarded  as  typical.  The  authors  state  thar 
the  objection  has  no  completely  satisfactory  answer  but  they  agree  that 
to  know  how  teachers  and  pupils  behave  during  observation  seems  better 
than  to  know  nothing  at  all  about  how  they  behave. 

Mitzel  and  Rabinowitz  (1953)  in  their  study  described  earlier  felt 

that  teacher  verbal  behavior  was  markedly  influenced  by  the  presence  of 

observers  in  the  classroom,  particularly  during  the  early  visits.   In 

addition  to  the  impressions  of  the  observers  that  initially  they  were 

watching  "tailor  made"  lessons,  they  use  logic  to  support  this  conclusion 

On  logical  grounds ,  it  appears  likely  that  the  net  effect 
of  "preparation  for  being  observed"  is  to  reduce  the 
differences  among  teachers  in  all  statement  categories 
and  to  have  a  general  depressing  effect  upon  the  total 
variance  for  each  category.  Though  it  cannot  be  proved 
conclusively,  this  is  a  plausible  explanation  for  the 
increase  in  the  among-teachers  mean  square  and  in  the 
total  variances  from  the  first  series  of  visits  to  the 
second.   (Mitzel  and  Rabinowitz,  1953,  p.  14) 
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Masling  and  Stern  (1969)  had  seven  observers  trained  in  the  use  of 
rating  scales.  Twenty-three  teachers  were  observed  for  a  period  of  two 
days  by  a  single  observer.   An  average  of  33  five-minute  observations 
were  made  over  the  two-day  period.  They  assumed  that  if  the  hypothesis 
that  the  effect  of  the  presence  of  the  observer  diminishes  gradually  over 
time  was  correct,  then  correlations  between  the  earliest  observations 
and  the  later  ones  should  be  lower  than  the  correlations  between  ob- 
servations at  a  later  period.  Data  were  reported  for  correlations  be- 
tween observations  1-3  versus  7-33,  4-6  versus  7-33,  and  7-9  versus 
10-33.  Of  the  ten  ratings  made,  only  two,  "class  climate-freedom"  and 
"class  climate-tension,"  produced  results  to  support  the  hypothesis. 
There  was  no  consistent  discernible  pattern  over  time.   Two  alternative 
conclusions  were  presented:  (a)  observer  influence  is  negligible;  the 
lack  of  consistency  over  time  was  found  because  the  variables  under  study 
occur  episodically;  or  (b)  the  effects  of  the  observer  are  more  complex 
than  has  been  foreseen  and  affect  various  aspects  of  teacher  and  pupil 
behavior  differentially. 

Samph  (1968)  made  tape  recordings  in  classrooms  both  without  the 
teacher's  knowledge  and  with  the  teacher's  knowledge.  Following  the  tape 
recording,  observers  collected  live  data  using  Flanders'  Interaction 
Analysis.  Data  were  given  on  five  variables:  amount  of  praise,  amount 
of  criticism,  total  teacher  acceptance  of  pupil's  ideas,  the  I/D  ratio 
(number  of  indirect/direct  teacher  behaviors)  and  the  i/d  ratio  (similar 
to  the  I/D  but  omitting  the  primarily  substantive  categories).   For  only 
the  first  two  variables  were  significant  differences  found  between  ob- 
servations made  with  and  without  an  observer  present  in  the  classroom. 
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With  the  presence  of  the  observer  the  amount  of  praise  increased  and 
the  amount  of  criticism  decreased  significantly.  None  of  the  other 
variables  showed  significant  change. 

The  two  studies  just  mentioned  illustrate  the  two  basic  paradigms 
by  which  reactive  effects  have  been  studied:   by  comparison  of  the 
effects  of  various  levels  of  obtrusiveness  of  observations  or  by  the 
study  of  behavioral  stability  over  time.   Johnson  and  Bolstad  (1973) 
review  the  literature  on  reactivity  in  naturalistic  observation  and  on 
the  basis  of  these  studies  discuss  four  sources  of  interference:  (1)  the 
conspicuousness  of  the  observer,  (2)  the  individual  differences  of  the 
subjects,  (3)  the  personal  attributes  of  the  observer  and  (4)  the  amount 
of  rationale  given  subjects  for  being  observed.  They  feel  there  can  be 
little  doiibt  that  the  entire  question  of  reactivity  has  been  inadequately 
researched  and  raise  the  point  that  not  only  may  reactivity  seriously 
limit  the  generalizability  of  naturalistic  observation  data  but  it  may 
interact  with  the  variables  under  study  to  confound  results . 


CHAPTER  III 


PROCEDURE 


Sample 

President  Johnson  initiated  the  Follow  Through  program  in  1968  to 
capitalize  upon  and  supplement  the  gains  made  by  disadvantaged  young- 
sters in  pre-school  programs  such  as  Head  Start.  As  in  Head  Start, 
Follow  Through  took  advantage  of  planned  variation;  that  is,  local  com- 
munities could  elect  to  be  self-sponsored  or  could  choose  to  be  spon- 
sored by  one  of  a  number  of  program  models  (Maccoby  and  Zellner,  1970). 
The  models  differed  widely  in  philosophy.   The  Becker-Englemann  model 
and  the  token  economy  of  Bushell  used  behavior  modification  techniques. 
Gordon's  parent  education  model  focused  on  parent  involvement  in  the 
motivation  and  education  of  their  children.   Nimnicht  stressed  the  re- 
sponsive environment  and  autotelic  learning  while  Gotkin  emphasized 
matrix  games.   Educational  Development  Center  (EDC)  sponsored  an  open 
education  model  based  on  the  British  Infant  School.  The  Bank  Street 
College  of  Education  approach  and  the  Tucson  early  education  model  com- 
pleted the  group  of  planned  variation  sponsors  whose  classrooms  were  ob- 
served in  the  present  study. 

Data  were  collected  in  289  classrooms  representing  the  eight  Follow 
Through  sponsors  described  above  and  a  number  of  comparison  (non-Follow 
Through)  classrooms.   Three  grade  levels  were  represented:  kindergarten; 
first  grade,  divided  into  an  entering-f irst  (i.e.,  a  first  grade  without 
a  kindergarten)  and  continuing-first;  and  second  grade.   A  summary  of 
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the  distribution  of  classrooms  by  sponsor  and  by  grade  level  is  given 
in  Table  1. 

TABLE  1 
DISTRIBUTION  OF  CLASSROOM  BY  SPONSOR  AND  BY  GRADE  LEVEL 


Grade 

Levels 

Sponsors 

K 

E-1 

C-1 

2 

Total 

Bank  Street 

9 

8 

10 

7 

34 

Becker-Englemann 

8 

9 

10 

U 

31 

E.D.C. 

8 

10 

6 

4 

28 

Gotkin 

12 

~ 

8 

~ 

20 

Nimnieht 

11 

8 

11 

7 

37 

Parent  Education 

8 

IC 

9 

6 

33 

Tucson 

8 

U 

9 

8 

29 

Bushell 

7 

5 

10 

— 

22 

Comparison 

15 

13 

18 

9 

55 

TOTAL 


86 


67 


91 


45 


289 


Data  Collection 

Systematic  classroom  observations  were  made  during  January,  February 
and  March  of  1971,  using  the  Florida  Climate  and  Control  System  (FLACCS) 
and  the  Teacher  Practices  Observation  Record  (TPOR).   Fourteen  graduate 
students  had  been  hired  in  the  summer  of  1970  to  be  observers  during 
the  winter  of  1971.   These  observers  were  trained  in  a  scheduled  credit- 
ed course  during  the  fall  quarter  at  the  University  of  Florida.   These 
14  observers  plus  four  permanent  members  of  the  evaluation  team  collected 
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the  observation  data.  Team  memberships  were  changed  periodically  during 
the  ten  weeks  of  observation  and  within  the  constraints  of  scheduling 
each  observer  was  assigned  to  as  many  sponsors'  classrooms  as  possible. 
The  number  of  classrooms  observed  each  week  by  sponsor  is  given  in 
Table  2. 

Both  FLACCS  and  the  TPOR  instruments  were  set  up  so  that  each  com- 
pleted instrument  contained  a  series  of  three  five-minute  observations: 

Series  1        Series  2         Series  3         Series  4 

Observation      Observation      Observation      Observation 
1    23      U    5    6       7    89       10   11  12 

During  a  total  day  in  the  classroom,  four  series  of  observations  were  made 
using  each  of  the  instruments,  two  series  by  one  observer  and  two  by  the 
other. 

The  rate  of  observing  was  dependent  on  the  length  of  the  school  day. 
If  a  kindergarten  class  was  three  hours  long,  the  rate  for  that  class- 
room would  be  four  five-minute  observations  per  hour.  The  rate  for  a 
six-hour  class  would  be  two  observations  per  hour.  Each  series ,  there- 
fore, represented  one-fourth  of  a  complete  school  day,  however  long  that 
day  might  be. 

At  the  end  of  the  day,  in  the  classroom,  each  of  the  two  observers 
completed  ratings  on  (1)  classroom  attitude  toward  observers ,  and  (2) 
attention  to  observers: 

(1)'  Classroom  attitude  toward  observers: 

.  Very                                          Friendly, 
Hostile    Up-tight Neutral Polite Open 
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(2)  Are  the  observers  in  the  classroom: 


A  Focus  of 

Attention A  Normal  Disruption Ignored 


Data  Analysis 

In  the  original  data  analysis ,  tallies  for  each  item  were  summed 
across  the  four  series  (12  observations)  for  each  teacher.   Means  and 
standard  deviations  for  each  item  were  calculated  across  all  teachers . 
Items  with  very  small  means  and/or  variances  were  then  either  eliminated 
or  pooled  with  related  items .  An  area  transformation  was  then  carried 
out,  item  by  item,  to  make  the  data  as  nearly  normally  distributed  as 
possible.  The  data  for  each  observation  system  were  then  factor  analyzed 
separately  using  principal  axis  factor  extraction  with  multiple  R  squared 
in  the  diagonal,  followed  by  varimax  rotation.  Nine  factors  were  identi- 
fied for  FLACCS  and  seven  for  the  TPOR.  The  composition  of  the  factors 
including  item  description  and  loadings  are  presented  in  Tables  B-1 
through  B-16  in  Appendix  B.   Incomplete  factor  scores  were  calculated 
for  each  of  the  teachers  on  each  of  the  factors  by  pooling  the  T-scores 
from  the  area  transformation  for  those  measures  which  loaded  +_  .UO  or 
above.  The  incomplete  factor  scores  of  the  nine  FLACCS  factors  and  the 
seven  TPOR  factors  for  each  of  the  289  teachers  in  the  study  were  then 
used  in  further  analyses  to  differentiate  teacher  and  pupil  behaviors 
across  the  eight  Follow  Through  programs  (and  comparison  classrooms)  and 
to  relate  those  classroom  behaviors  to  the  mean  academic  growth  of  pupils 
(Soar,  1973). 
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For  the  present  study  the  identical  procedure  of  obtaining  incom- 
plete factor  scores  was  followed,  except  that,  instead  of  summing  the 
data  for  each  classroom  across  all  12  observations  (four  series  of  three 
observations),  the  sums  for  each  of  the  four  series  were  used.   In  effect 
this  gave  four  scores  for  each  item  for  each  classroom  instead  of  one . 
Each  score  represented  three-five-minute  time  slices  of  behavior  over 
one-quarter  of  the  school  day.   The  data  were  T-scored  by  area  trans- 
formation and  incomplete  factor  scores  calculated.  The  end  result  was 
four  repeated-measures  for  each  teacher  on  each  of  the  nine  factors  of 
FLACCS  and  the  seven  factors  of  the  TPOR. 

Reliability 

Incomplete  factor  scores  ov3r  four  periods  of  the  day  for  289  teach- 
ers were  analyzed  using  a  one-way  repeated  measures  analysis  of  variance. 
An  overall  reliability  score  for  each  factor  was  computed  using  Hoyt's 
formula  (4)  (1955,  p.  52). 

MS^  -  MS 
I     e 


11       MS 

where  MS  and  MS  designate  the  mean  square  for  individuals  and  for  error 
respectively.  All  four  factor  scores  for  each  teacher  entered  into  the 
overall  reliability  computation  for  each  factor. 

Two  scores  for  each  teacher  based  on  data  collected  by  the  same  ob- 
server were  analyzed  using  a  repeated-measures  analysis  of  variance. 
The  score  calculated  from  formula  (4)  using  only  these  data,  represents 
within- observer  reliability.   Since  there  were  two  observers  in  the 
classroom,  each  recording  two  series  of  observations,  two  such  intra- 
observer  reliabilities  were  calculated. 
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A  similar  analysis  of  variance  was  used  to  analyze  the  data  of 
series  2  and  series  3  done  by  different  observers.   Using  formula  (H) 
again,  an  inter-observer  reliability  score  was  calculated. 

Intra-observer  and  inter-observer  reliabilities  were  calculated 
using  only  two  of  the  four  time  periods  for  each  classroom.  Each  of 
these  reliability  figures  was  adjusted  by  the  Spearman-Brown  formula 
(Cronbach,  1960,  p.  131)  for  obtaining  the  reliability  expected  from 
twice  as  much  data. 

Stability  of. Classroom  Behavior 

The  repeated-measures  analyses  of  variance  using  all  four  incomplete 
factor  scores  for  each  classroom  provided  data  on  the  stability  of  class- 
room behaviors  for  each  of  the  nine  factors  of  FLACCS  and  each  of  the 
seven  factors  of  the  TPOR. 

Observer  Effects 

The  ratings  of  classroom  attitude  and  attention  to  observers,  made 
by  each  of  the  observers  at  the  end  of  each  day  of  observation,  were 
summed  to  produce  one  score  for  each  classroom  on  each  rating  scale . 
Classrooms  were  blocked  on  the  attitude  and  attention  ratings  and  the  in- 
complete factor  scores  for  each  FLACCS  and  TPOR  factor  were  subjected  to 
a  two-way  repeated  measures  analysis  of  variance.  This  was  done  to  de- 
termine whether  perceived  differences  in  classroom  attitude  or  attention 
to  observers  were  accompanied  by  any  significant  differences  in  classroom 
behaviors  or  by  any  significant  interaction  effects. 


CHAPTER  IV 


RESULTS 


Incomplete  factor  scores  for  nine  FLACCS  factors  and  seven  TPOR 
factors  were  generated  for  289  Follow  Through  and  non-Follow  Through 
classrooms  over  four  periods  of  time-of-day.  The  factor  scores  were 
used  in  further  analyses  in  three  related  substudies  investigating  (1) 
the  reliability  of  the  observation  instruments ,( 2 )  the  stability  of 
classroom  behavior,  and  (3)  the  effect  of  the  observer  in  the  classroom. 
Results  are  reported  separately  for  each  of  these  areas  of  investigation. 

Reliability 

Reliabilities  derived  from  several  repeated  measures  analyses  of 
variance  are  given  in  Table  3  for  the  nine  factors  from  the  Florida 
Climate  and  Category  System  and  in  Table  U  for  the  seven  factors  of  the 
Teacher  Practices  Observation  Record. 

Overall  reliabilities  are  based  on  four  series  of  observations  (four 
repeated  measures)  for  each  of  the  289  teachers  in  the  study.   The  within- 
observer  or  intra-observer  reliabilities  are  based  on  the  two  series  of 
observations  (two  repeated  measures)  done  by  the  same  observer.   Within- 
observer  reliabilities  are  broken  into  those  collected  early  in  the  day 
by  the  first  observer  and  those  collected  later  in  the  day  by  the  second 
observer  in   each  of  the  289  classrooms .   The  between-observer  or  inter- 
observer  reliabilities  are  based  on  the  middle  two  series  of  observations 
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(two  repeated  measures)  done  by  two  different  observers  in  each  of  the 
289  classrooms. 

Each  intra-observer  and  inter-observer  reliability  figure  is  equiv- 
alent to  the  correlation  between  the  two  observations  adjusted  by  the 
Spearman- Brown  formula  (Cronbach,  1960,  p.  131)  for  obtaining  the  relia- 
bility coefficient  of  the  two  series  of  observations  together.   Since 
each  intra-observer  and  inter-observer  reliability  figure  as  presented 
was  based  on  only  two  of  the  four  series  of  observations,  each  of  those 
figures  was  again  adjusted  by  the  Spearman-Brown  formula  to  obtain  the 
reliability  expected  from  twice  as  much  data.  .  Adjusted  reliability 
scores  are  presented  in  Tables  3  and  4  in  parentheses. 

Unadjusted  intra-observer  reliabilities  range  from  coefficients  of 
.62  to  .86  for  FLACCS  factors  and  from  .61  to  .82  for  TPOR  factors. 
Scores  adjusted  by  the  Spearman-Brown  formula  are  higher,  ranging  from 
.77  to  .93  for  FLACCS  factors  and  .76  to  .90  for  TPOR  factors.   The 
intra-observer  reliabilities  measure  the  consistency  of  recordings. with- 
in observers . 

Unadjusted  inter-observer  reliabilities  range  from  coefficients  of 
.38  to  .70  for  FLACCS  factors  and  from  .37  to  .68  for  TPOR  factors. 
Scores  adjusted  by  the  Spearman-Brown  formula  range  from  .55  to  ,82  for 
FLACCS  factors  and  from  .54  to  .81  for  TPOR  factors.  The  inter-observer 
reliabilities  measure  the  consistency  of  recordings  across  observers . 

Overall  reliabilities  range  from  .64  to  .84  on  the  FLACCS  factors 
and  from  .62  to  .82  on  the  TPOR  factors. 


31 

Stability  of  Classroom  Behavior 
Florida  Climate  and  Control  System  (FLACCS) 

Mean  factor  scores  on  nine  FLACCS  factors  over  four  periods  of  the 
day  for  the  289  Follow  Through  classrooms  are  presented  in  Table  5. 
Also  included  are  the  F's  from  a  one-way  repeated-measures  analysis  of 
variance  of  the  data  for  each  factor .  Graphs  of  the  data  appear  as 
Figure  1'. 

Significant  differences  in  classroom  behavior  across  four  periods 
of  the  day  characterize  all  FLACCS  factors  except  for  Factor  9,  Teacher 
Positive  Affect.   Factor  1,  consisting  of  behaviors  representing  strong 
cpntrol  and  negative  affect  on  the  part  of  the  teacher,  shows  a  signifi- 
cant increase  over  time  along  with  the  behaviors  of  Factor  7,  Pupil 
Negative  Affect.   Classrooms  also  tend  to  move  toward  more  free  choice 
in  the  later  part  of  the  day  (Factor  2).   The  last  half  of  the  day  tends 
to  show  less  Teacher-Pupil  Supportive  Behavior  (Factor  3),  Gentle  Control 
(Factor  5),  Work  Without  the  Teacher  (Factor  6),  and  Teacher  Attention 
in  a  Task  Setting  (Factor  8).   In  general  the  data  support  the  belief 
that  most  of  the  "work"  of  the  classrooms  tends  to  be  done  earlier  in 
the  day. 

Teacher  Practices  Observation  Record  (TPOR) 

Mean  factor  scores  over  four  periods  of  the  day  for  the  289  Follow 
Through  classrooms  are  presented  in  Table  6  along  with  the  F  from  a  one- 
way repeated-measures  analysis  of  variance  for  each  of  the  seven  TPOR 
factors.  Graphs  of  the  data  appear  as  Figure  2. 

There  were  highly  significant  changes  over  the  day  for  three  TPOR 
factors:   Factor  1  -  Convergent  Teaching,  Factor  "4  -  Undifferentiated 
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FIGURE  1.  Graphs  of  mean  factor  scores  for  2S9  classrooms  across  fovx 
series  of  observaticr.s  during  the  day  on  each  of  nine  FLACCS 
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FIGURE  2.  Graphs  of  mean  factor"  scores  for  289  classrooms  across  four 
series  of  observations  during  the  day  on  each  of  seven  TPOR 
factors. 
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Teaching,  and  Factor  5  -  Pupil  Free  Choice  vs.  Teacher  Structured  Acti- 
vity.  Most  convergent  teaching  occurs  in  the  early  part  of  the  day  fall- 
ing off  rapidly  toward  the  end  of  the  day.   The  most  differentiated 
teaching  occurs  in  the  early  part  of  the  day  with  undifferentiated  teach- 
ing high  later  in  the  day.   Teacher  structured  activities  tend  to  occur 
in  the  morning  with  increasing  freedom  of  pupil  choice  over  the  course 
of  the  day. 

Factor  3  -  Teacher  Discourages  Exploration  is  also  significant  with 
regard  to  change  over  the  day.  The  items  of  which  this  factor  is  com-  • 
posed  are  all  odd-numbered  items  on  the  TPOR,  i.e.,  they  are  behaviors 
attributed  philosophically  to  the  traditional  teacher.  All  the  items 
represent  an  attempt  to  keep  the  student  "on  the  right  track"  in  a  teach- 
ing-learning situation.   This  behavior  increased  over  most  of  the  day  and 
declined  over  the  final  series  of  observations . 

Factor  2  -  Experimental  Teaching  and  to  a  large  extent  Factor  7  - 
Exploration  of  Ideas  attempt  to  measure  more  experimental  types  of  be- 
havior, perhaps  those  behaviors  which  fit  into  the  open-classroom,  child- 
centered  approaches  to  early  education.  Analysis  of  the  measurement  of 
such  behaviors  does  not  disclose  significant  changes  over  the  time  of 
day . 

In  general,  the  data  from  the  TPOR  agree  with  the  data  from  the 
FLACCS  in  showing  that  most  of  the  "work"  (in  the  traditional  sense)  of 
the  classroom  tends  to  be  done  in  the  first  half  of  the  school  day  with 
students  getting  increased  freedom  of  choice  as  the  day  goes  on. 

Observer  Effects 
The  distribution  of  the  sum  of  the  ratings  by  two  observers  of  the 
scale  on  classroom  attitude  is  given  for  the  289  Follow  Through  Classrooms 
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in  Table  7.  The  distribution  for  the  scale  on  attention  to  observer  is 
given  in  Table  8. 

Incomplete  factor  scores  for  289  classrooms,  blocked  (1)  by  the 
classroom  attitude  scores  and  (2)  by  the  attention  to  observer  scores, 
were  subjected  to  two-way  analyses  of  variance.  Results  are  presented 
separately  for  each  of  the  two  scales . 

TABLE  7 

SUM  OF  RATINGS  BY  TWO  OBSERVERS 
ON  THE  SCALE  OF  CLASSROOM  ATTITUDE 


Group 


Frequency 


Sum  of 

Rating 

Two  Ratings 

Description 

(1)  Hostile 

3 

4 

(2)  Uptight 

5 

6 

(3)  Neutral 

7 

8 

(4)  Polite 

9 

10 

(5)  Friendly,  ( 

13 
20 
30 
70 
77 
79 


Classroom  Attitude 

Mean  factor  scores  for  the  six  groups  of  classrooms  blocked  by 
ratings  on  classroom  attitude  are  given  in  Table  9  for  the  nine  FLACCS 
factors  and  in  Table  10  for  the  seven  TPOR  factors.   Highly  significant 
differences  were  found  among  classrooms  with  different  ratings  of 
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TABLE  8 

SUM  OF  RATINGS  BY  TWO  OBSERVERS 
ON  THE  SCALE  OF  ATTENTION  TO  OBSERVER 


Sum  of  Rating 

Group         Frequency       Two  Ratings        Description 


(1)  A  Focus  of 
2  Attention 

1  10  3 

2  15  H 

3  31^  5    . 

(3)  A  Normal 
'^  63  6  Disruption 

5  67  7 

6  62  8 

7  25  9 

8  12  10       (5)  Ignored 


classroom  attitude  on  FLACCS  factors  1,  5,  7,  and  9.  The  up-tight  to 
neutral  end  of  the  attitude  scale  was  associated  with  more  strong  con- 
trol (and  teacher  negative  affect),  less  gentle  control,  more  pupil 
negative  affect  and  less  teacher  positive  affect.   The  friendly  end  of 
the  scale  was,  conversely,  associated  with  less  strong  control,  more 
gentle  control,  less  pupil  negative  affect  and  more  teacher  positive 
affect. 

Significant  interactions  between  groups  and  time  of  day  occurred  for 
FLACCS  factors  7  and  8  and  TPOR  factors  1  and  6.   Mean  factor  scores  for 
each  group  over  the  four  periods  of  the  day  are  given  in  Tables  C-1  to 
C-4,  respectively,  in  Appendix  C. 
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41 
Attention  to  Observers 

Of  the  TPOR  factors,  only  TPOR-2,  Experimental  Teaching,  was  signi- 
ficantly related  to  grouping  or  classroom  attitude.   Classrooms  that 
were  rated  most  open  and  friendly  were  those  in  which  the  most  experi- 
mental teaching  occurred. 

A  similar  procedure  was  followed  for  blocking  classrooms  on  the 
basis  of  attention  to  observers  as  was  followed  for  the  ratings  of  class- 
room attitude.   Results  from  a  two-way  analysis  of  variance  for  the  nine 
FLACCS  factors  are  given  in  Table  11  and  for  the  seven  TPOR  factors  in 
Table  12. 

Again,  significant  dif f ei ences  were  found  among  classrooms  with 
different  ratings  of  attention  to  observers  on  FLACCS  factors  1,  5,  7, 
and  9.   Greater  amounts  of  attention  to  observers  tend  to  be  associated 
with  more  negative  affect  and  strong  control  on  the  part  of  the  teacher 
(FLACCS-1)  and  with  more  negative  affect  on  the  part  of  the  pupils 
(FLACCS-7).   Little  attention  to  observers  tends  to  be  related  to  gentle 
control  (FLACCS-5)  and  teacher  positive  affect  (FLACCS-9).   These  results 
parallel  the  findings  from  classroom  attitude  ratings  although  they  are 
neither  as  consistent  nor  as  striking. 

More  of  the  behaviors  of  TPOR-6,  the  unnamed  factor,  occur  in  the 
extremes  of  the  attention  scale  than  occur  in  the  middle  ranges .   The 
difference  is  significant  but  uninterpretable .   There  are  four  signifi- 
cant interactions  of  groups  by  time  of  day  in  TPOR  factors  1,  3,  4,  and 
5 .   Mean  factor  scores  for  each  of  the  eight  groups  over  four  periods 
of  the  day  are  given  for  TPOR  factors  1,  3,  4,  and  5  in  Tables  C-5,  C-6, 
and  C-8,  respectively,  in  Appendix  C. 
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CHAPTER  V 


DISCUSSION 


Results  have  been  reported  for  three  related  substudies  investi- 
gating (1)  the  reliability  of  the  FLACCS  and  the  TPOR,  (2)  the  stabili- 
ty of  classroom  behaviors  measured  by  the  two  instruments,  and  (3)  the 
effect  of  the  observers  in  the  classroom.  Each  of  the  substudies  will 
be  discussed  separately. 

Reliability 
Intra-Observer  and  Inter-Observer  Reliabilities 

In  the  reliability  data  reported  in  the  last  chapter  for  nine  FLACCS 
factors  and  seven  TPOR  factors,  inter-observer  or  between-observer  re- 
liabilities are  often  noticeably  poorer  than  intra-observer  or  within- 
observer  reliabilities.  In  all  but  three  cases,  adjiasted  intra-observer 
reliability  figures  are  higher  than  the  actual  overall  reliability, 
while  adjusted  inter-observer  reliability  figures  are  lower.  Even  in 
the  three  cases  where  this  does  not  hold  true  —  FLACCS-2,  FLACCS-5,  and 
TPOR-1  —  it  is  inter-observer  reliability  which  has  decreased  the  actual 
overall  reliability  figure,  rather  than  intra-observer  reliability.   In 
any  series  of  four  observations,  two  each  by  two  observers ,  only  two 
combinations  of  series  are  possible  for  establishing  intra-observer  re- 
liabilities —  the  two  series  done  by  one  observer  and  two  series  done 
by  the  other.   Six  combinations  are  possible,  however,  with  four  series 
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of  observations.  Only  one  combination  —  series  two  and  series  three  — 
was  selected  to  produce  a  measure  of  inter-observer  reliability  in  this 
study.  Three  possible  additional  combinations  were  not  calculated. 
Where  the  average  of  the  expected  reliabilities  does  not  agree  with 
the  overall  reliability,  the  discrepancy  is  due  to  inter-observer  re- 
liability figures  which  are  not  reported. 

Comparison  with  Other  Studies 

Inter-observer  reliability  figures  reported  here,  although  in  part 
a  measure  of  agreement  between  observers,  are  not  comparable  to  those 
measures  of  observer  agreement  in  which  different  observers  are  tallying 
the  same  classroom  behaviors  at  the  same  time.  The  inter-observer  reli- 
abilities reported  here  are  based  on  different  observers  collecting  data 
at  two  different  times  in  each  of  289  classrooms.  Therefore,  comparisons 
cannot  be  made  meaningfully  between  the  reliabilities  reported  in  the 
present  study  and,  for  instance,  the  percent  agreement  scores  based  on 
a  total  instrument  as  reported  by  Stallings  (1972),  where  two  observers 
were  paired  in  eight  classrooms  to  collect  data  simultaneously. 

Intra-observer  reliabilities  likewise  have  no.  equivalent  counterpart 
in  reported  studies.   Intra-observer  reliabilities  as  reported  by  Brown 
(1968),  for  instance,  are  based  on  observations  made  by  the  same  ob- 
servers of  behaviors  from  the  same  videotape  a  year  apart.   Intra-ob- 
server reliability  as  reported  in  the  present  study  is  based  on  observa- 
tions by  the  same  observer  of  the  same  classroom  at  two  different  times . 
Actual  differences  in  behavior  are  reflected  in  the  figures .  Meaningful 
comparisons  across  the  two  studies  cannot  be  made. 
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Implications  from  the  Reliability  Figures 

Certain  recommendations  may  be  made  in  an  attempt  to  increase  reli- 
ability. Since  inter-observer  reliability  figures  were  lower  than  intra- 
observer  f  igijires ,  additional  training  time  might  be  spent  observing  those 
behaviors  for  which  the  inter-observer  reliabilities  were  lower.  An 
increase  in  reliability  might  also  be  gained  by  increasing  the  number  of 
observations,  although  the  corresponding  increase  in  cost  must  also  be 
considered.  Changes  in  the  observation  instruments,  themselves,  might 
also  help  to  increase  reliability. 

From  the  data  presented  it  is  obvious  that  certain  groups  of  be- 
haviors are  recorded  with  more  overall  reliability  than  others.  Behaviors 
related  to  teacher  negative  affect  and  strong  control  (FLACCS-1) ,  pupil 
negative  effect  (FLACCS-7),  teacher  positive  affect  (FLACCS-9),  or  free 
versus  more  structured  (FLACCS-2,  TPOR-5,  and  TPOR-7)  teaching  styles  are 
recorded  with  more  reliability  than  those  behaviors  related  to  teacher 
gentle  control  (FLACCS-5),  teacher  attention  in  a  task  setting  (FLACCS-8), 
undifferentiated  teaching  (TPOR-4)  or  the  unnamed  factor  (TPOR-6).  A 
closer  look  at  these  factors  reveals  an  interesting  finding.  Those  fac- 
tors with  reliabilities  in  the  .80's  are  composed  of  more  items  of  be- 
havior than  those  factors  with  reliabilities  in  the  .50's.  Tables  13 
and. 14  show  that  higher  reliabilities  of  the  FLACCS  and  TPOR  tend  to  be 
associated  with  larger  numbers  of  items.   If  validity  data  (a  signifi- 
cant correlation  with  pupil  gain,  for  instance)  indicated  the  importance 
of  a  factor  and  an  attempt  to  increase  reliability  was  deemed  worthwhile, 
one  method  of  increasing  reliability  could  be  to  add  to  the  observation 
instrument  items  which  might  be  related  to  the  factor.  Adding  items  — 
without  increasing  the  total  number  of  observations  —  might  increase 
reliability  without  an  appreciable  increase  in  cost. 
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TABLE  13 

OVERALL  RELIABILITY  AND  THE  NUMBER 
OF  ITEMS  FOR  EACH  OF  NINE  FLACCS  FACTORS 


FLACCS                 Overall  Number 

Factor               Reliability  of  Items 

1  .84  23 
'9                      .84  10 

7  .83  13 

2  .81  14 

3  .78  .           11 

4  .74  4 
6                      .70  3 

5  .69  6 

8  .64  5 

TABLE  14 

OVERALL  RELIABILITY  AND  THE  NUMBER 
OF  ITEMS  FOR  EACH  OF  SEVEN  TPOR  FACTORS 


TPOR  Overall  Number 

Factor  Reliability  of  Items 


7  .82  16 

5  .80  20 

1  .78  7 

3  .75  7 

2  .71  13 

6  .68  4 

4  .62  3 
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The  differences  in  reliability  figures  among  FLACCS  and  TPOR  factors 
demonstrate  again  the  importance  of  calculating  reliability  figures  for 
the  variables-  used'  in  research  rather  than  calculating  one  reliability 
figure  for  the  instrument  as  a  whole.   This  point  has  already  been 
made  by  Soar  (1973)  and  by  Johnson  and  Bolstad  (1973). 

Perhaps  a  further  point  needs  to  be  made.   Reliabilities  based  on 
data  gathered  in  Follow  Through  classrooms  may  perhaps  be  higher  than 
reliabilities  using  the  same  instruments  in  less  variable  classroom  set- 
tings. The  wide  variety  of  approaches  in  the  planned-variation  phase  of 
Follow  Through  increases  the  opportunity  for  variance  across  classrooms 
to  be  significantly  greater  thi-n  variance  within.  The  definition  of  re- 
liability as  used  in  this  study,  is  based  on  the  concept  of  the  average 
difference  between  two  measures  obtained  in  the  same  classroom  being 
smaller  than  the  average  difference  between  two  measures  obtained  in 
different  classrooms.   The  possibility  of  higher  reliability  figures  is 
increased  as  differences  across  classrooms  increase.  It  is  difficult  to 
imagine  a  broader  variety  of  kindergarten,  first-  and  second-grade  class- 
rooms than  exists  in  the  present  sample. 

Stability  of  Classroom  Behaviors 
That  much  teacher  behavior  changes  significantly  throughout  the 
school  day  is,  perhaps,  not  surprising.  Most  kindergarten,  first-  and 
second-grade  teachers  have  schedules  or  lesson  plans  which  partition  the 
day  into  periods  of  different  activities.   Indeed,  what  may  be  surpris- 
ing is  the  similarity  of  the  reported  means  across  time  of  day  for  the 
factors  of  the  FLACCS  and  the  TPOR. 
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Although  there  were  statistically  significant  differences  in  means 
across  time  for  many  classroom  behaviors,  those  differences  do  not  appear 
to 'be  large.  Significance  was  reached  because  of  relatively  small  dif- 
ferences which  were  consistently  replicated  in  other  classrooms.   Thus, 
significance  was  reached  because  much  of  the  within-classroom  variance 
could  be  accounted  for  by  the  time  of  day. 

Despite  significant  changes  in  behavior  over  time,  those  behaviors 
may  be  measured  reliably.  The  most  significant  changes  in  behavior  in 
the  present  study  occurred  for  teacher  negative  affect  and  strong  control 
(FLACCS-1),  pupil  negative  affect  (FLACCS-7)  and  convergent  teaching 
(TPOR-1).  Overall  reliabilities  for  these  factors  were  .8^+,  .83,  and  .78, 
indicating  that  instability  of  classroom  behaviors  during  the  school  day 
need  not  necessarily  be  a  barrier  to  good  reliability.  Lack  of  stability 
of  classroom  behavior  does  suggest,  however,  the  importance  of  the  time 
of  day  in  an  observation  schedule.  Predictable  or  lawfiil  changes  in  some 
behaviors,  e.g.,  structured  teaching  activities  or  negative  affective 
behaviors,  seem  to  occur  at  least  at  the  kindergarten,  first  and  second 
grade  levels  of  the  schools  in  the  present  study. 

Observer  Effects 


Results  of  two-way  analyses  of  variance  —  time  of  day  by  groups 
blocked  on  classroom  attitude  or  attention  to  observers  —  were  reported 
in  Chapter  M-.   Those  analyses  were  done  to  determine  whether  perceived 
differences  in  classroom  attitude  or  attention  to  observers  were  accom- 
panied by  any  significant  differences  in  classroom  behaviors.   The  most 
consistent  of  the  significant  differences  among  groups  of  classrooms 
blocked  by  ratings  on  classroom  attitude  and  attention  to  observers 
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occurred  in  factors  1,  5,  7,  and  9  of  the  FLACCS.   The  most  significant 
associations  involved  FLACCS-1  and  FLACCS-9  and  the  ratings  on  classroom 
attitude.  The  behaviors  of  FLACCS-1  are  instances  of  teacher  negative 
affect  and  strong  control  while  behaviors  of  FLACCS-9  are  indicators  of 
teacher  positive  affect.   The  association  of  positive  or  negative  teach- 
er affective  measures  and  ratings  of  classroom  attitude  are  not  sur- 
prising even  though  significant.  Much  of  the  information  on  which  a 
rating  of  classroom  attitude  toward  observers  is  made,  is  based  on  the 
teacher's  affective  behavior. 

Samph  (1968),  in  a  study  reported  earlier,  found  that  with  an  ob- 
server present  in  the  classroom,  the  amount  of  criticism  on  the  part  of 
the  teacher  diminished  significantly  and  the  amount  of  praise  increased. 
If,  as  Masling  and  Stern  (1969)  hypothesized,  the  effect  of  the  presence 
of  an  observer  diminishes  gradually  over  time,  then  one  might  expect 
that  the  amount  of  criticism  would  increase  and  the  amount  of  praise  de- 
crease over  the  course  of  a  day  of  observation  as  teachers  and  students 
adjusted  to  the  presence  of  the  observer.   From  the  present  study,  it  is 
obvious  that  teacher  negative  affect  (of  which  teacher  criticism  is  a 
part)  did,  in  fact,  increase  over  the  day.  Positive  affect  (of  which 
teacher  praise  is  a  part)  did  not  decrease,  however.  Perhaps  the  pre- 
sence of  an  observer  differentially  affects  positive  and  negative  affec- 
tive behaviors.  Perhaps  changes  in  the  amount  of  negative  affective 
behavior  for  teachers  and  students  over  time  are  the  result  of  something 
other  than  the  presence  of  observers  —  fatigue  and  irritability  perhaps . 
Certainly,  differences  in  the  variables  under  consideration  and  dif- 
ferences in  their  measurement  have  complicated  comparisons  across  the 
studies. 
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Of  the  TPOR  factors,  only  TPOR-2,  Experimental  Teaching,  was  signi- 
ficantly related  to  the  ratings  on  classroom  attitude.  Perhaps  teachers 
who  operate  in  open,  experimental  classrooms  can  more  easily  be  friendly 
or  open  in  their  attitudes  toward  observers . 

Implications  for  Further  Research 

Research  using  systematic  observation  would  be  strengthened  by  the 
reporting  of  reliability  figures  as  demonstrated  in  this  study.  They 
emphasize,  again,  the  wisdom  of  calculating  reliability  not  on  the  total 
instrument,  but  on  those  variables  used  in  the  analysis  of  the  data. 
Reliability  figures  point  out  the  areas  of  relative  deficiency  within 
the  observation  instrument.   They  allow  one  to  determine  which  areas 
need  special  emphasis  during  the  training  phase  of  a  study,  they  also 
direct  attention  to  areas  where  instrument  revision  might  help  to  bolst- 
er reliability.  They  give  the  best  overall  indication  of  the  accuracy 
of  the  measurements  used  in  the  data  analysis  phase  of  a  study.  Routine 
reporting  of  reliabilities  computed  in  the  data  analysis  phase  of  studies 
using  systematic  observation  would  allow  greater  opportunity  for  compari- 
son of  instruments  as  well  as  selection  and  refinement  of  instruments 
suitable  for  continued  use. 

Inter-observer  reliability  figures  have  implications  for  research 
design  as  well.  Where  between-observer  differences  are  known  to  occur, 
individual  observers  should  not  be  confounded  with  different  experi- 
mental programs  or  treaments  but  should  be  distributed  across  all 
programs . 
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Results  of  the  substudy  on  the  stability  of  classroom  behaviors 
have  implications  for  the  timing  of  classroom  observations  within  the 
overall  research  design. 

And,  finally,  further  study  is  needed  to  investigate  observer 
effects  on  classroom  behaviors.   The  issue  of  reactivity  has  not  been 
adequately  researched. 


CHAPTER  VI 
SUMMARY  AND  CONCLUSIONS 

Observable  behaviors  are  the  focus  of  systematic  observation  in- 
strument's. However,  the  data  produced  to  measure  classroom  behaviors 
through  systematic  observation  are  the  result  of  at  least  two  categories 
of  behavior:   (1)  that  classroom  behavior  which  is  being  observed,  and 
(2)  the  behavior  of  an  observer  which  results  in  tallies  on  the  observa- 
tion instrument.   In  addition,  when  more  than  one  observer  is  involved 
in  data  collection,  one  must  be  concerned  about  (3)  the  behavior  of 
additional  observers.  When  more  than  one  classroom  is  involved  in  the 
data  collection,  one  is  primarily  concerned  about  (4)  those  behaviors 
which  differentiate  one  classroom  from  another. 

Questions  about  consistency  arise  with  regard  to  each  of  the  four 
categories  of  behavior:   (1)  Is  a  teacher  consistent  in  her  behavior? 
Is  the  class  consistent  in  its  behavior?   (2)  Is  an  observer  consistent 
in  recording  what  he  sees?   (3)  Are  all  observers  equivalent?  To  what 
extent  are  they  the  same  measuring  instrument?   (4)  Is  the  observation 
instrument  capable  of  revealing  systematic  differences  between  classrooms 
and  teachers? 

In  the  present  study,  answers  to  the  questions  raised  above  were 
sought  (1)  by  analyzing  the  stability  of  classroom  behaviors,  (2)  by  com- 
puting intra-observer  (or  within-observer)  reliabilities,  (3)  by  estimat- 
ing inter-observer  (or  between-observer)  reliabilities,  and  (U)  by 
computing  overall  reliabilities  on  each  of  the  measures  of  behavior. 
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The  data  under  consideration  were  collected  using  two  systematic 
observation  instruments:   the  Florida  Climate  and  Control  Systems,  FLACCS; 
and  the  Teacher  Practices  Observation  Record,  TPOR.  Data  from  four 
series  of  observations,  representing  four  periods  throughout  the  school 
day,  were  collected  in  289  Follow  Through  and  comparison  classrooms. 
Item  data  were  T-scored  by  area  transformation  and  reduced  to  incomplete 
factor  scores  on  nine  FLACCS  factors  and  seven  TPOR  factors.  From  the 
data  of  a  one-way  analysis  of  variance  on  each  of  the  FLACCS  and  TPOR 
factors,  reliabilities  were  calculated.  Results  showed:   (1)  that  many 
teacher  behaviors  change  significantly  over  the  course  of  a  day  while 
others  remain  quite  stable,  (2)  that  intra-observer  reliabilities  were, 
in  general,  high,  (3)  that  inter-observer  reliabilities  were  not  as  high 
as  intra-observer  reliabilities  and  that  the  inter-observer  reliabilities 
of  some  factors  were  considerably  better  than  others,  and,  (H)  that  over- 
all reliabilities  for  each  of  the  factors  were  generally  in  the  .70's 
and  .80's.  Four  of  the  16  factors  had  overall  reliabilities  in  the  .60's. 
Significant  changes  in  classroom  behavior  over  the  course  of  a  day  did 
not  preclude  high  overall  reliability  in  the  measurement  of  those 
behaviors. 

Results  of  several  two-way  analyses  of  variance  demonstrated  that 
some  perceived  differences  in  classroom  attitude  toward  observers  or 
attention  to  observers  are  accompanied  by  a  significant  differences  in 
behaviors  measured  by  FLACCS  or  TPOR  factors.   The  most  significant  re- 
lationships were  between  teacher  positive  or  negative  affective  be- 
haviors and  ratings  of  classroom  attitude.  More  data  are  needed  to 
properly  assess  the  effect  of  observers  on  classroom  behavior. 
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Implications  for  further  research  were  drawn  from  the  data  of  the 
study.   Routine  calculation  and  reporting  of  reliability  data  was 
recommended  in  all  studies  using  systematic  observation. 
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APPENDICES 


APPENDIX  A 


TPOR  AND  FLACCS 


TEACHER  PRACTICES  OBSERVATION  RECORD 


DIRECTIONS 


The  Teacher  Practices  Observation  Record  provides  a  framework  for 
observing  and  recording  the  classroom  practices  of  teacher.  Your  role 
as  an  observer  is  to  watch  and  listen  for  signs  of  the  62  teacher  prac- 
tices listed  and  to  record  whether  or  not  they  were  observed,  WITHOUT 
MAKING  JUDGMENTS  AS  TO  THE  RELATIVE  IMPORTANCE  OR  RELEVANCE  OF  THOSE 
PRACTICES. 

There  are  three  (3)  separate  ten-minute  observation  and  marking 
periods  in  each  30-minute  visit  to  the  teacher's  classroom.  These  are 
indicated  by  the  column  headings  I,  II,  and  III.  During  period  1,  spend 
the  first  five  minutes  observing  the  behavior  of  the  teacher.   In  the 
last  five  minutes  go  down  the  list  and  place  a  check  (   )  mark  in  Column 
I  beside  all  practices  you  saw  occur.   Leave  blank  the  space  beside 
practices  which  did  not  occur  or  which  did  not  seem  to  apply  to  this 
particular  observation.  Please  consider  every  practice,  listed,  mark  it 
or  leave  it  blank.  A  particular  item  is  marked  only  once  in  a  given 
column,  no  matter  how  many  times  that  practice  occurs  within  the  ten- 
minute  observation  period.  A  practice  which  occurs  a  dozen  times  gets 
one  check  mark,  the  same  as  an  item  which  occurs  only  once. 

Repeat  this  process  for  the  second  ten-minute  period,  marking  in 
Column  II.  Repeat  again  for  the  third  ten-minute  period,  marking  in 
Column  III.  Please  add  the  total  number  of  check  marks  recorded  in 
Columns  I,  II,  and  III  for  each  teacher-  practice  and  record  in  the  col- 
umn head  TOT.   There  may  be  from  0  to  3  total  check  marks  for  each  item. 


Name  of  Teacher 


month         day        year 


city         state 


School 


Name  of  Observer- Judge 


Grade  Subject 


FIGURE  A-1:   THE  TEACHER  PRACTICES  OBSERVATION  RECORD 
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TEACHER  PFcAClICKS  OBSERVATICtI  RECORD 


TOT        I 

II 

JJL 

TEACKER  PRACTICES 
A.        NATURE  or  THE  SiTaATIOM 

1.      T  occupie;.   center  of  aftention 

j          i     2.      T  mi'.'re:.   D  renter  of  at  Li'!:t:i  on. 

c  \~i  i  "^ 

3.      T  n'.ak3S   son"   thv.ig  ar.  a   t.-'-in;-   center  of   p'a  attdition. 

4.      T  r.akc'.-   dciiij;  cc:i'fthiiij;   c£:ru;i'  oi    p'r;   atLsntion. 

! 

i 

5.      T  ho.-j  p  ;.pe!id   time  waitin;',,   wdtchinp.,    list-Jni.r.n;. 

!          ;     b.      T  natj  p  p-ir'"icipate  actively. 

1          i          '     7.      T  r<.:ni3Lr:s   a^loof  oi'  C-iZactoa    fron  ::'s  -:ct.  iv.i  ■;;iPL-, . 

1 

8.      T   ioiii':   or-  po.' ticipatos    in  v':>   activi  t  i  fs  . . 
^.      T    ii^.^ufar,c3   or-  pr'-ve:.t=   p   fioni  oxpr-s?  -  ;>f   self   frcej.y. 
10.      T  enroui  c?f.s  o   to  ri:;pt<...s3  .^.elf    freoly. 

i 

1           1                                              B.         NATURE  or  THE   J-SOf-lXM 

■ 

1          ■  11.      T  or^aiii^os   Itiariii.ii^  arounrl  Q  pcsod  by   T. 

■ 

!          1 

12.  T  or,-, -jr. iter.   ic:>rnin?,  around  u's   own  probliif  or  Q. 

13.  T  prcvori-j   s'ltu-rition  which  c-ijses   p  'if>;ht  or  perplexity. 

i 

1          ■  I't.      T   ii'vclvos   D  in  uncrrta.i.n  or   incoanJetc  situ^iiori. 

■           i          ;  15.      T  iiro.n'i   p  a».ay   fro:^!  "tiL-rd"   C  cr  proMcin . 

j          !          i  lb.     T  Jradn  p   to  Q  or  pr^M"-.  v.-hich   "c:r.!i:i^s"   Ui^. 

1 

,  17.   .T   cr.-.tih-jsi2>ii   idealised,   ioa;;.s-jrinz,   oi''  "pr'^tiy''  aspects 
)                       »                                        '                       3 »            «           y           t 

1             of  topic. 

1  18.     T  eiiipba^iies  realistic,   diEConcorting,  or  ''ij.jly"  aspects 
I             of  topic. 

19.     T  acks  Q  that  p  caa  answer  only   if  he  studied  the 
lesr-cn. 

20.     T  asks  Q  tnat   is   not  I'eadily  ans>werat.ie  by  study  of 
lesson. 

C.        DEVKI,0!';-!ENT  OF  IDEAS 

21.     T  accents  oavy  one  answer  as  baing  correct. 

1 

! 

22.     T  ponriits   p  to  suggest  additional  or  aitern^Ative 
ancwerc . 

1 

7'-'.      T  oxpccts   p   to  cone   uo  with  answer  T  has   in  nii.-d. 

f 
1 

24.     T  asks  p  to   judge  cocnparative  value  of  ar-£w-=i's  or 
suk-r.estions. 

i           ■          1  25.      T  oipocts:   p  to   "k.'.cv;"   rather  trwn   Co  p.v.ess   anrv.-c-r  to  0. 

26.      T  encouragfjs   p  to  guess  or  hypothesise  about   the 
unknovm  cr  untested. 

27.     T  accepts  only  answers  or  suggestions  closely  related 
to  tojjic. 

1 

28.      T  entertains   even  "wild"  or  far-fotchcd  .suggestion  of  p. 

1 

20.  ,   T  lets  p  "g'it   by"  with  opinionated  or'  stereotyped 
answef. 

30.      T  ask;;   p  to  support  answer  or  opinion  with   evidence. 

FIGURE  A-1    (Continued ) 
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TOT 

I   II 

Ill 

D.   US"  or  SUBJECT  MATTE?. 

1 

31.   T  collects  and  an<?.lyzoE  sui;ieci;  mailer  for  D. 

1     1 

32.  T  hac  p  xake  his  own  collection  and  analysis  of 
subject  matter. 

1    ; 

33.  "l    provisos  D  with  dntcilc-l  facts  and  irifo'.'riatior,. 

!   j 

34.   T  has  p  rind  detail,-J  faits  iind  informa'tion  on  hie  otm. 

1   1 

35.   T  rosier.  h';avi  Iv  on  toxU.or.k  ar.   source  u*^  infcrTr.aciori. 

■  1   i 

3S.   T  na.-,i,9  a  wice  rar.p.u  cS   iTiforn-.atic.T  r.aXf.z'.ol.   avcilab'e. 

I   i 

37.   !  ac(.-cpts  c::d    uifS  indr.cu--':'ro  i:ii^c-nn3t  i~r; . 

i 

1 

" 

38.   T  l.eJps  p  discover  and  con-ect  factual  crioru  and 
inaccuracies. 

i 

39.   T  penr.its  formation  of  misconoeptions  and  o/er- 
g-sne)alizations. 

1 
1 

40.   T  q'-iu..ticns  miscouc-ptions,  faulty  locic,  unwarranted 
ccnclasions. 

E.    EVAr,n.'.TICM 

i   1 

41.  T  ;-Tsses  judprnrint  on  n's  b'-havior  or  work. 

1 

4?.   T  withholds  -iud?.rnont  on  o's  b'-havior  or  vjc.rk . 

43.   T  stops  p  from  going  a.hoad  with  plan  v/'aich  T  knows 
will  lail. 

44.  T  'jncourage?  p  to  out  his  idoar.  to  a  t?:;t. 

4d.   T  i;.-..nodiately  reinfcrcoa  p's  apswei-  cs   rijht"  cr 
.  "wrong . " 

4b.   T  has  p  decide  when  Q  has  been  answered  satisfactorily. 

47.  T  ajks  another  p  to  give  answer  if  one  p  fails   to 
ansver  quickly. 

1 

48.  T  asks  n  to  evaluate  his  own  work. 

40.   T  provides  answer  to  p  who  soerT:G  confused  or  puzzled. 

50.  T  givfis  p  tiT.e  to  sit  and  think,  :r.ull  things  ever. 

F.   DIFrERZKTKi.TICJf 

1 

51.   7  has  all  p  workint;  at  sar^e  ta^k  at  sar.e  tiu.e. 

52.   T  has  different  D  wcrkir.t'  at  different  tasks. 

1 

53.  T  holds  all  p  responsible  for  certain  material  to  be 
learned. 

54.   T  has  p  work  indepondentlv  on  what  concerns  D. 

55.  T  evaluates  work  of  all  p  bv  a  set  standard. 

65.  T  evaluates  work  of  different  p  by  different 
standards . 

G.   HOTIVATIO!?,  COfTTPOL 

57.   T  motivates  p  with  privileges,  prizes,  grades. 

58.   T  motivates  p  with  intrinsic  value  of  ideas  or 
activity. 

59.  T  a:'pfoaches  subiect  natter  in  direct,  business-like  wav. 

60.   T  aporoaches  subiect  tratter  in  indii-oct,  informal  way. 

1 

61.   T  iir.r.oses  external  dlici.iiinarv  control  on  p. 

! 

62.   T  encourages  self -discipline  on  part  of  p. 

FIGURE  A-1    (Continued. . . ) 
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INSTITUTE  FOR  THE  DEVELOPMENT  OF  HUMAN  RESOURCES 
University  of  Florida 
Gainesville,  Florida 
November  25,  1970 


FLACCS* 


Florida  Climate  and  Control  System 


Program 
City 


School 
Grade 


Teacher 
Date 


Observer 
Series 


*This  is  an  experimental  form  which  should  not  be  cited  or  used  without 
permission  of, the  developers. 


FIGURE  A-2.   THE  FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 
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NEGATIVE  AFFECT 


Verbal             Teacher 

Nonverbal 

c. 

Tot 

1 

2 

3 

C. 

Tot 

1 

2 

3 

10 

Says  "stop  it,"  etc. 

10 

Waits  for  child 

li. 

Uses  t^^eatening  tone 

11 

Frowns 

12 

Rejects  child 

12 

Points,  shakes  finger 

13 

Criticizes,  blames 

13 

Pushes  or  pulls,  holds 

iu 

Warns 

lu 

Shows  di^ust 

is 

Yells 

15 

Takes  material 

16 

Scolds,  humiliates 

16 

Refuses,  to  respond  to  child 

n 

Other 

17 

Other 

18 

Code  Involvement 

Verbal             Pupil                  Nonverbal 

19 

Says  "No,"  "I  won't,"  etc. 

18 

Makes  face,  frowns 

20 

Teases 

19 

Pouts,  withdraws 

21 

Laughs 

20 

Uncooperative,  resistant 

22 

Tattles 

21 

Stamps,  throws,  slams 

23 

Commands  or  demands 

■?7 

Interferes,,  threatens 

2"* 

Makes  disparaging 

23 

Takes,  damages  property 

25 

Demands  attention 

2U 

Picks  at  child 

26 

Makes  someone  "feel  small" 

25 

Pushes. or  pulls,  holds 

27 

Finds  fault 

26 

Hits,  hurts 

28 

Threatens 

27 

Is  left  out 

29 

Other 

28 

Other 

30 

Code  Involvement 

Verbal             Teacher                   Nonverbal 

31 

Says  "Thank  yoUj^'  etc. 

29 

Accepts  favors  for  self 

32 
33 

Agrees  with  child 

30 

Waits  for  child. 

Supports  child 

31 

Gives  individual  attention 

3'4 

Gives  individual  attention 

32 

Warm,  congenial 

3b 

Warm,  congenial 

33 

Listens  carefully  to  child 

36 

Praises  child- 

34 

Smiles,  laughs,  nods 

37 

Develops,  "we  feeling" 

35 

Pats,  hugs,  etc. 

38 

Is  enthusiastic 

36 

Sympathetic 

39 

Other 

37 

Other 

40 

Code  Involvement 

Verbal              Pi. 

pil                    Nonverbal 

W 

Says  "Thank  you,"  etc. 

38" 

Helpful ,  shares 

H7 

Sounds  friendly 

39 

Leans  close  to  another 

U3 

Agrees  with  another 

■40 

Chooses  another 

UU 

Initiates  contact 

«1 

Smiles,  laughs  with  another 

4!) 

Offers  to  share,  cooperate 

■♦2 

Pats,  hugs  another 

Ub 

Supports  another 

"43 

Agreeable,  cooperative 

fl 

Is  enthusiastic 

U4 

Enthusiastic 

ta 

Praises  another 

U5 

Horseplay 

H9 

Helps  another 

U6 

Other 

bO 

Other 

147 

(Continued  over) 

bl 

Code  Involvement 

CODE  INVOLVEMENT 

0.  None  involved 

1.  Few  involved 

2.  Up  to  1/2  the  class 

3.  More  than  half 


FIGURE  A-2   (Continued...) 
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c. 

Tot 

1 

2 

3 

TEACHER             C . 

Tot 

1 

2 

3 

PUPIL 

lU 



Teacher  Central               10 

PuDil  Central 

Leads  s Lnging, games, storytm 

11 

Pupil  --  no  choice 

v/ 

Moves  freely  among  pupils 

12 

Pupil  —  linitrd  choice 

J.J 

Withdraws  from  class 

13 

Pupil  —  free  choLoe 

Uses  blackboard,  A-V  Equip. 

- 

lb 



Ignores, refuses  to  attend  P. 

lu 

P'Seat  work  w/o  teacher 

_^ 

Attends  P.  briefly 

15 

(*Seat  work  with  teacher 





Attends  P.  closely 

■ 





Attends  P.  in  succession 

lb 

(*Works,  plays  w.  much  supv. 

1.. 

Attends  simultaneous  activ. 

17 

(*Works,  plays  w.  little  supv. 



VtKBAL  CONTROL 

18 

(^Resists,  disobeys  directions 

Praises 

19 

21 

Asks  for  status 

20 

n 

Suggests,  guides 

21 

Follows  routine  w/o  reminder 



Feedback,  cites  reason 

22 

Reoorts  rule  to  another 

Questions  for  refltiv.thot 

23 

Tattles 



, 

Correct  w/o  criticism  (SM) 

2t 

Gives  infonnation 

, 



Questions  for  control 

25 

Gives  direction 





. 

Questions  states  beh.rule 

26 

Gives  reason 

M 

1 

1 

Directs  with  reason 

27 

Speaks  aloud  w/o  permission 

1 



Directs  w/o  reason 

28 

Engages  in  out-of-bounds  beh. 

Uses  time  pressure 

29 

Collaborates  w.  teacher 

Jij 

Call  child  by  name  (WS) 

30 

Task  related  movement 



Interrupts  Pupil, cuts  off 

31 

Aimless  wandering 

Warns 

32 

Fantasy 

Supv.  p.  closely, imblizes. 

33 

3b_ 

^ 

__ 

Criticizes 

3U 

' 

Parallel  play  or  work 





Orders,  commands 

3b 

Works, plays  collaboratively 





Scolds,  punishes 

35 

Works,  plays  competitively 

JB 





Uses  firm  tone 

37 

Seeks  reassurance,  support 

, 



Uses  sharp  tone 

38 

Shows  pride 







39 

Shows  fear, shame, humiliation 



. 

MO 

Shows  aoathv 

NONVERBAL  CONTROL                               wnPK  r.pniipc; 

■Jl 

— 

— 

Tolerates  deviant  beh . 

m 

- 



Pupil  as  individual 

12 

Nods,  smiles  for  control 

U3 



Structured  groups  w/o  t. 





Positive  facial  feedback 

LiU 

" 

Free  groups 

Uses  "body  English" 

Hb 

Gestures 

■ib 

Gives  tangible  reward 

SOCIALIZATION 





Touches,  pats  (gentle) 

MS 

Almost  never 

uq 

— 

— 

Ho Ids, pushes ,spanks( harsh) 

M6 

_ 

Occasionally 

50 

— 



Signals,  raps 

-LlA 

^ 

bi 

Shhh:  Shakes  head 

b:^ 

Glares,  frowns 

.  MATERIALS 

U8 

Structure  T.  behavior 

149 



Structure  P.  behavior 

PUPIL  INTEREST  ATTENTION 


1 (Rank  1  low  to  5  high 


FIGURE  A-2  (Continued...) 
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APPENDIX  B 


ITEM  LOADINGS  AND  DESCRIPTIONS  FOR  FLACCS 
FACTORS  1  THROUGH  9  AND  TPOR  FACTORS  1  THROUGH  7 


69 


TABLE  B-1 
FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 
Factor  1  -  Strong  Control 


Item 


Loading 


Description 


11  .71 

12  .73 

13  .70 

18  .  53 

19  .51 

20  .77 
26  .56 
32  .46 
39  .41 

50  .62 

51  .82 

52  .87 

55  .50 

56  .70 

57  .78 

58  .73 

69  .74 

70  .68 

71  .45 

72  .61 

82  .89 

83  .52 

1  -.42 
Eigenvalue  =  11,69 


Teacher  warns 

Teacher  criticizes 

T  orders,  commands 

T  holds,  pushes,  spanks  (harsh) 

T  says,  "Shhhl"  Shakes  head 

T  glares ,  frowns 

Pupil  resists,  disobeys  directions 

P  engages  in  out-of-bounds  behavior 

P  shows  apathy 

Level  3,  T  verbal  control 

Level  4,  T  verbal  control 

Level  5 ,  T  verbal  control 

Level  3,  T  nonverbal  control 

T  says  "Stop  it,"  etc. 

T  uses  threatening  tone 

T  criticizes,  blames 

T  frowns 

T  points,  shakes  finger 

P  makes  face,  frowns 

P  uncooperative,  resistant 

Total  T  negative 

Total  P  negative 

Free  groups 


^Source:  Soar,  1973 
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TABLE  B-2A 
FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 
Factor  2  -  Pupil  Free  Choice  vs.  No  Choice 


Item 


Loading 


Description 


3 
22. 
23 
33 
314 
36 
42 
44 
81 
85 

2 
10 
21 
43 

Eigenvalue  =  6.95 


.41 
.62 
.60 
.56 
.63 
.59 
.67 
.59 
.43 
.50 

.65 
.54 
.80 
.49 


T  attends  p  closely 

Pupil  limited  choi!ce 

Pupil  free  choice 

Task  related  movement 

P  uses  play  object  as  itself 

P  works ,  plays  collaboratively 

Free  work  groups 

Frequent  socialization 

P  pats ,  hugs  another 

Total  p  positive 

Teacher  central 

T  directs  without  reason 

Pupil  no  choice 

Almost  no  socialization 


TABLE  B-3* 

FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 

Factor  3  -  Teacher-Pupil  Supportive  Behavior 


Item 


Loading 


Description 


5  .58 

■  7  .43 

37  .59 

38  .57 
48  .44 
62  .68 

■  67  .65 
77  .65 
.79  .55 
80  .57 
85  .41 

Eigenvalue  =  5.73 


T  suggests ,  guides 

T  corrects  without  criticism  (SM) 

P  seeks  reassurance,  support 

P  shows  pride 

Level  1,  T  verbal  control 

T  supports  child 

P  agrees  with  another 

T  waits  for  child  (positive) 

P  helpful,  shares 

P  leans  close  to  another 

Total  P  positive 


•Source:  Soar,  1973 
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TABLE  B-4''« 
FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 
Factor  4  -  Nonverbal  Gentle  Control 


Item 


Loading 


Description 


16  .64 

17.  .47 

54  .65 

29  -.48 

Eigenvalue  =3.04 


T  gestures 

T  touches,  pats,  Cgentle) 

Level  2,  T  nonverbal  control 

P  gives  information, 


TABLE  B-5A 

FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 

Factor  5  -  Gentle  Control 


Item 


Loading 


Description 


14 
15 
49 
53 

Eigenvalue  =3.44 


.44 
.65 
.68 
.54 
.65 
.70 


T  questions  for  reflective  thought 
T  questions  for  control 
T  nods,  smiles  for  control 
T  positive  facial  feedback 
Level  2,  T  verbal  control 
Level  1,  T  nonverbal  control 


•'Soiorce:   Soar,  1973 
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TABLE  B-6'^ 

FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 

Factor  6  -  Work  Without  Teacher 


Item 


Loading 


Description 


21+ 

.58 

35 

,41 

41. 

.62 

Eigenvalue 

=   2.63 

P  seatwork  without  teacher 

Parallel  work  or  play 

Structured  groups  without  teacher 


TABLE  B-7>'« 

FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 

Factor  7  -  Pupil  Negative  Affect 


Item 


Loading 


Description 


30  .43 

32  .63 

59  .70 

60  .65 

61  .69 

71  .43 

72  ,48 

73  .57 

74  .70 

75  .66 

76  .56 
83  .75 

43  -.41 


Eigenvalue  =  6.65 


P  gives  directions 

P  engages  in  out-of-bounds  behavior 

P  teases 

P  commands  or  demands 

P  threatens 

P  makes  face,  frowns 

P  uncooperative,  resistant 

P  interferes ,  threatens 

P  takes,  damages  property 

P  picks  at  child 

P  pushes  or  pulls ,  holds 

Total  P  negative 

Almost  no  socialization 


''Source:   Soar,  1973 
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Item 


TABLE  8-8=^' 

FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 

Factor  8  -  Teacher  Attention  in  a  Task  Setting 


Loading 


3 

.68 

25 

.47 

1^0 

.46 

46 

.43 

47 

.77 

Eigenvalue  = 

=   3.13 

Description 


T  attends  P  closely 

P  works,  plays  with  much  supervision 

Pupil  as  individual  (work  groups) 

Materials  Structure  P  behavior 

T  closeness  of  attention 


Item 


TABLE  B-9" 
FLORIDA  CLIMATE  AND  CONTROL  SYSTEM 
Factor  9  -  Teacher  Positive  Affect 


Loading 


1  .52 

27  .49 

63  .46 

64  .70 

65  .67 

66  .56 
68  .62 
78  .55 

84  .78 

85  .46 

Eigenvalue  =  5.23 


Description 


Pupil  interest  attention  rating 

P  obeys  directions 

T  gives  individual  attention 

T  warm,  cogenial 

T  is  enthusiastic 

T  sounds  friendly 

P  is  enthusiastic  (verbal) 

T  smiles,  laughs,  nods 

Total  T  positive 

Total  P  positive 


^Source:   Soar,  1973 
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Item 


TABLE  B-10" 

TEACHER  PRACTICES  OBSERVATION  RECORD 

Factor  1  -  Convergent  Teaching 


Loading 


Description 


1 
21 

23 

25 

27 

45 

47 

Eigenvalue  =4.42 


.42        T  occupies  center  of  attention 
.70        T  accepts  only  one  answer  as  being 

correct 
.82        T  expects  p  to  come  up  with  answer 

T  has  in  mind 
.70        T  expects  p  to  "know"  rather  than  to 

guess  answer  to  Q 
.74        T  accepts  only  answers  or  suggestions 

closely  related  to  topic 
.45        T  immediately  reinforces  p's  answer 

as  "right"  or  "wrong" 
.52        T  asks  another  p  to  give,  answer  if 

one  p  fails  to  answer  quickly 


*Source:   Soar,  1973 
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TABLE  B-11* 

TEACHER  PRACTICES  OBSERVATION  RECORD 

Factor  2  -  Experimental  Teaching 


Item 


Loading 


Description 


14 

16. 

18 

24 

26 

32 

34 

38 

40 

42 

46 

48 
50 

Eigenvalue  =  5.37 


.47 

T 

.67 

T 

.48 

T 

.43 

T 

.46 

T 

.51 

T 

.56 

T 

.71 

T 

.61 

T 

.46 

T 

.42 

T 

.52 

T 

.65 

T 

involves  p  in  uncertain  or  incomplete 
situation 

leads  p  to  Q  or  problem  which 
"stumps"  him 

emphasizes  realistic,  disconcerting, 
or  "ugly"  aspects  of  topic 
asks  p  to  judge  comparative  value 
of  answers  or  suggestions 
encourages  p  to  guess  or  hypo- 
thesize about  the  unknown  or 
untested 

has  p  make  his  own  collection  and 
analysis  of  subject  matter 
has  p  find  detailed  facts  and 
information  on  his  own 
helps  p  discover  and  correct 
factual  errors  and  inaccuracies 
questions  misconceptions,  faulty 
logic,  unwarranted  conclusions 
withholds  judgment  on  p's  behavior 
or  work 

has  p  decide  when  Q  has  been 
answered  satisfactorily 
asks  p  to  evaluate  his  own  work 
gives  p  time  to  sit  and  think, 
mull  things  over 


*Source:   Soar,  1973 
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Item 


TABLE  B-12* 

TEACHER  PRACTICES  OBSERVATION  RECORD 

Factor  3  -  Teacher  Discourages  Exploration 


Loading 


Description 


9  .52 

13.  .66 

15  .65 

35  .^b 

43  .59 

49  .52 

55  .64 

Eigenvalue  =  3.91 


T  discourages  or  prevents  p  from 

expressing  self  freely 
T  prevents  situation  which  causes 

p  doubt  or  perplexity 
T  steers  p  away  from  "hard"  Q 

or  problem 
T  relies  heavily  on  textbook  as 

source  of  information 
T  stops  p  from  going  ahead  with  plan 

which  T  knows  will  fail 
T  provides  answer  to  p  who  seems 

confused  or  puzzled 
T  evaluates  work  of  all  p  by  a. set 

standard 


Item 


TABLE  B-13A 
TEACHER  PRACTICES  OBSERVATION  RECORD 
Factor  4  -  Undifferentiated  Teaching 


Loading 


51  .92 
53  .57 

52  -.92 

Eigenvalue  =2.71 


Description 


T  has  all  p  working  at  same  task 

at  same  time 
T  holds  all  p  responsible  for 

certain  material  to  be  learned 
T  has  different  p  working  at 

different  tasks 


*Source:   Soar,  1973 
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TABLE  B-li4='' 
TEACHER  PRACTICES  OBSERVATION  RECORD 
Factor  5  -  Pupil  Free  Choice  vs.  Teacher  Structured  Activity 


Item 


Loading 


2 

.61 

4 

.74 

6 

.65 

8 

.68 

10 

.42 

12 

.60 

32 

.52 

36 

.47 

1+14 

.51 

54 

.59 

60 

.68 

1 

-.50 

3 

-.68 

5 

-.67 

11 

-.61 

19 

-.50 

25 

-.45 

31 

-.48 

33 

-.47 

59 

-.69 

Description 


T  makes  p  center  of  attention 

T  makes  doing  something  center  of 

p's  attention 
T  has  p  participate  actively 
T  joins  or  participates  in  p's 

activities 
T  encourages  p  to  express  self 

freely 
T  organizes  learning  around  p's  own 

problem  or  Q 
T  has  p  make  his  own  collection  and 

analysis  of  subject  matter 
T  makes  a  wide  range  of  informational 

material  available 
T  encourages  p  to  put  his  ideas  to  a 

test 
T  has  p  work  independently  on  what 

concerns  p 
T  approaches  subject  matter  in 

indirect,  informal  way 

T  occupies  center  of  attention 

T  makes  some  thing  as  a  thing  center 

of  p's  attention 
T  has  p  spend  time  waiting,  watching, 

listening 
T  organizes  learning  around  Q  posed 

by  T 
T  asks  Q  that  p  can  answer  only  if  he 

studied  the  lesson 
T  expects  p  to  "know"  rather  than  to 

guess  answer  to  Q 
T  collects  and  analyzes  subject  matter 

for  p 
T  provides  p  with  detailed  facts  and 

information 
T  approaches  subject  matter  in  direct, 

business-like  way 


Eigenvalue  =8.39 


*Source:   Soar,  1973 
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Item 


TABLE  B-15* 

TEACHER  PRACTICES  OBSERVATION  RECORD 

Factor  6  -  Unnamed 


Loading 


Description 


20  .52 

29.  .44 

56  .60 

61  .45 

Eigenvalue  =  2.18 


T  asks  Q  that  is  not  readily 

answerable  by  study  of  lesson 
T  lets  p  "get  by"  with  opinionated 

or  stereotyped  answer 
T  evaluates  work  of  different  p 

by  different  standards 
T  imposes  external  disciplinary 

control  on  p 


'Source:   Soar,  1973 
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TABLE  B-16''- 
TEACHER  PRACTICES  OBSERVATION  RECORD 
Factor  7  -  Exploration  of  Ideas  vs.  Textbook  Learning 


Item 


Loading 


10  .60 

22-  .71 

24  .1+3 

26  .59 

28  .58 

32  .43 

36  .45 

44  .43 

58  .56 
60  ^50 
62  .47 

35  .43 

41  .42 

45  .41 
57  .43 

59  .46 

Eigenvalue  =  6.02 


Description 


T  encourages  p  to  express  self  freely 

T  permits  p  to  suggest  additional  or 
alternative  answers 

T  asks  p  to  judge  comparative  value 
of  answers  or  suggestions 

T  encourages  p  to  guess  or  hypo- 
thesize about  the  unknown  or 
untested 

T  entertains'  even  "wild"  or  far- 
fetched suggestion  of  p 

T  has  p  make  his  own  collection  and 
analysis  of  subject  matter 

T  makes  a  wide  range  of  informational 
material  available 

T  encourages  p  to  put  his  ideas  to 
a  test 

T  motivates  p  with  intrinsic  value 
of  ideas  or  activity 

T  approaches  subject  matter  in 
indirect ,  informal  way 

T  encoiirages  self -discipline  on 
part  of  p 

T  relies  heavily  on  textbook  as  a 

source  of  information 
T  passes  judgment  on  p's  behavior 

and  work 
T  immediately  reinforces  p's  answer  as 

"right"  or  "wrong" 
T  motivates  p  with  privileges ,  prizes, 

grades 
T  approaches  subject  matter  in 

direct,  business-like  way 


•'Source:   Soar,  1973 


APPENDIX  C 


DATA  ON  SIGNIFICANT  INTERACTIONS  BETWEEN  TIME 

OF  DAY  AND  GROUPS  BASED  ON  RATINGS  OF 
CLASSROOM  ATTITUDE  OR  ATTENTION  TO  OBSERVERS 
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TABLE  C-1 

MEAN  FACTOR  SCORES  SHOWING  THE  INTERACTION* 

OF  CLASSROOM  ATTITUDE  GROUPINGS  AND  TIME  OF 

DAY  FOI^  FLACCS  FACTOR  7,  PUPIL  NEGATIVE  AFFECT 


Time 

of  Day 

Re 
M< 

Groups 

1 

2 

3 

4 

3W 

5  an 

1 

51, 

.05 

52, 

.35 

55, 

.86 

54. 

.50 

53, 

.44 

2 

49, 

.80 

52, 

.00 

51, 

.48 

52, 

.79 

51, 

.52 

3 

49, 

.89 

49, 

.60 

50, 

.75 

52. 

,06 

50, 

.57 

1+ 

49, 

.42 

50, 

.30 

50, 

.67 

50, 

.55 

50, 

.24 

5 

48, 

.47 

50, 

.13 

49. 

.82 

49, 

.58 

49, 

.50 

6 

49, 

.86 

50, 

.83 

50, 

.44 

50, 

.81  ■ 

50, 

.49 

Column 
Mean 

49, 

.44 

50, 

.53 

50, 

.68 

50, 

.85 

^F  =  2.62;  p  <  .01 

TABLE  C-2 

MEAN  FACTOR  SCORES  SHOWING  THE  INTERACTION- 
OF  CLASSROOM  ATTITUDE  GROUPINGS  AND  TIME  OF 
DAY  FOR  FLACCS  FACTOR  8,  TEACHER  ATTENTION  IN  A  TASK  SETTING 


Time 

of  Day 

Re 

M< 

Groups 

1 

? 

3 

4 

3W 

;an 

1 

49, 

.34 

48, 

.11 

50, 

.85 

49, 

.20 

49, 

.37 

2 

50, 

.99 

50, 

.99 

48, 

.16 

48, 

.92 

49, 

.76 

3 

48, 

.44 

48, 

.89 

48, 

.77 

50, 

.64 

49, 

.18 

4 

50, 

.10 

51, 

.30 

51. 

.05 

50, 

.69 

50, 

.78 

5 

48, 

.99 

50, 

.55 

49, 

,55 

49, 

.54 

49, 

.66 

6 

49, 

.99 

51, 

.51 

49, 

.82 

49, 

.44 

50, 

.19 

Column 
Mean 

49, 

.63 

50, 

.74 

49. 

.87 

49, 

.84 

*F  =  1.76;  p<  .05 
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TABLE  C-3 

MEAN  FACTOR  SCORES  SHOWING  THE  INTERACTION- 
OF  CLASSROOM  ATTITUDE  GROUPINGS  AND  TIME  OF 
DAY  FOR  TPOR  FACTOR  1,  CONVERGENT  TEACHING 


Time 

of 

Day 

Re 

Me 

Groups 

1 

2 

3 

4 

5an 

1 

50 

77 

50 

.72 

46 

79 

45 

24 

48 

38 

2 

50 

55 

50 

.08 

49 

57 

48 

91 

49 

78 

3 

50 

51 

50 

.53 

50 

95 

50 

10 

50 

52 

4 

51 

34 

52 

.15 

51 

07 

49 

26 

50 

95 

5 

49 

73 

50 

.38 

49 

05 

48 

01 

49 

29 

6 

50 

35 

50 

.19 

49 

38 

48 

43 

49 

59 

Column 
Mean 

50 

47 

50 

.77 

.49 

76 

48 

.53 

*F  =  1.75;  p 

<  .05 

TABLE 

C-i 

+ 

MEAN  FACTOR  SCORES  SHOWING  THE  INTERACTION* 

OF  CLASSROOM  ATTITUDE  GROUPINGS  AND  TIME  OF 

DAY  FOR  TPOR  FACTOR  6,  UNNAMED 


Time  of  Day 

Re 
Me 

Groups 

1 

? 

3 

4 

jan 

1 

51 

29 

50 

.65 

48 

62 

48 

15 

49 

68 

2 

49 

39 

50 

54 

51 

18 

51 

06 

50 

54 

3 

49 

34 

49 

.39 

49 

66 

49 

33 

49 

43 

4 

49 

55 

50 

.29 

50 

04 

49 

.63 

49 

87 

5 

49 

27 

49 

.22 

50 

47 

49 

44 

49 

60 

6 

50 

32 

50 

54 

50 

88 

50 

.34 

50 

.52 

Column 
Mean 

49 

73 

50 

.01 

50 

.36 

49 

.77 

*F  =1.96;  p.<  .05 
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TABLE  C-5 

MEAN  FACTOR  SCORES  SHOWING  THE  INTERACTION- 

OF  ATTENTION  TO  OBSERVER  GROUPINGS  AND  TIME 

OF  DAY  FOR  TPOR  FACTOR  1,  CONVERGENT  TEACHING 


Time 

of  Day 

R< 
M« 

Groups 

1 

2 

3 

4 

jan 

1 

54 

21 

54 

17 

49 

66 

45 

63 

51 

17 

2  ■ 

50 

58 

50 

91 

48 

71 

48 

70 

49 

73 

3 

51 

01 

50 

95 

50 

16 

48 

48  . 

50 

.15 

4 

50 

32 

50 

19 

49 

00 

48 

62 

49 

53 

5 

49 

45 

50 

64 

48 

87 

47 

57 

49 

13 

6 

50 

66 

50 

41 

50 

81 

49 

96 

50 

.46 

7 

49 

94 

50 

99 

51 

70 

48 

.54 

50 

29 

8 

52 

19 

52 

56 

49 

65 

48 

.73 

50 

.78 

Coliimn 

Mean  50.47  50.77  49.76  48.58 


*F  =   2.05;   p<:    .01 
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TABLE  C-6 

MEAN  FACTOR  SCORES  SHOWING  THE  INTERACTION- 
OP  ATTENTION  TO  OBSERVER  GROUPINGS  AND  TIME 
OF  DAY  FOR  TPOR  FACTOR  3,  TEACHER  DISCOURAGES  EXPLORATION 


Time 

of  Day 

Re 
M( 

Groups 

L 

2 

3 

4 

3W 

2an 

1   . 

51 

.79 

50 

.91 

49, 

.03 

*47 

.51 

49 

.81 

2 

51 

.18 

52, 

.15 

51, 

.39 

51, 

.41 

51, 

.53 

3 

49, 

.96 

49, 

.86 

50, 

.56 

50, 

.24 

50, 

.15 

4 

50, 

.37 

50, 

.65 

50, 

.38 

50, 

.96 

50, 

.59 

5 

50, 

.11 

50, 

.44 

51, 

.07 

50, 

.14 

50, 

.44 

6 

50, 

.01 

49, 

.95 

51, 

.23 

50, 

.60 

50, 

.45 

7 

49. 

.44 

51, 

.27 

50, 

.42 

49, 

.88 

50, 

.25 

8 

49, 

.90 

51, 

.01 

48, 

.93 

50, 

.06 

49, 

.98 

Column 
Mean 

50, 

,18 

50. 

,52 

50. 

,69 

50, 

,38 

*F  =  1.59;  p<  .05 
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TABLE  C-7 

MEAN  FACTOR  SCORES  SHOWING  THE  INTERACTION- 
OF  ATTENTION  TO  OBSERVER  GROUPINGS  AND  TIME 
OF  DAY  FOR  TPOR  FACTOR  4,  UNDIFFERENTIATED  TEACHING 


Time 

of  Day 

Re 
Me 

Groups 

1 

I 

3 

4 

;an 

1 

53, 

.60 

49, 

.30 

47, 

,94 

47, 

.57 

49, 

.60 

2 

51, 

.23 

52, 

.15 

50. 

,79 

50, 

.67 

51. 

.21 

3 

51, 

.09 

48, 

.55 

52. 

,10 

50, 

.36 

50, 

.53 

4 

50, 

,75 

50, 

.01 

50. 

,14 

51, 

.38 

50, 

.57 

5 

50, 

.17 

49, 

.09 

51. 

,58 

51, 

.97 

50. 

,70 

6 

48, 

.86 

47, 

.99 

50. 

.21 

52, 

.38 

49, 

,86 

7 

50, 

.38 

48, 

.57 

50, 

.35 

52, 

.85 

50, 

.54 

8 

52, 

.25 

50, 

.39 

49, 

.67 

52, 

.17 

51, 

,12 

Column 

Mean  50.40  49,18  50.68  51.60 


*F  =  1.66;   p<    .05 
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TABLE  C-8 

MEAN  FACTOR  SCORES  SHOWING  THE  INTERACTION-  OF  ATTENTION 

TO  OBSERVER  GROUPINGS  AND  TIME  OF  DAY  FOR  TPOR  FACTOR  5, 

PUPIL  FREE  CHOICE  VERSUS  TEACHER  STRUCTURED  ACTIVITY 


Time 

of  Da}! 

Re 

)W 

Groups 

1 

2 

2 

i 

^ 

^ 

Mean 

1 

49, 

,72 

49. 

,09 

52. 

,01 

53. 

,89 

51, 

,18 

2 

50, 

.74 

50-. 

,74 

50. 

,73 

51, 

,88 

51, 

.02 

3 

49, 

.78 

50, 

,59 

50. 

,58 

52. 

,43 

50, 

.84 

4 

49, 

.45 

49, 

.87 

50, 

.57 

50, 

.50 

50, 

.10 

5 

49, 

.59 

49, 

.85 

50. 

,23 

50, 

.97 

50, 

.16 

6 

49, 

.97 

50, 

.67 

50, 

,38 

49, 

.89 

50, 

.23 

7 

51, 

.03 

50. 

.92 

49, 

.27 

51, 

.36 

50, 

.65 

8 

50 

.08 

48, 

.87 

51, 

,52 

53, 

.01 

50 

.87 

Column 

Mean 

49 

.88 

50, 

.19 

50, 

.44 

51, 

.08 

*F  =  1.83;  p<  .05 
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